/**
@mainpage Remake, a build system that bridges the gap between make and redo.
As with make, remake uses a centralized rule file, which is
named Remakefile. It contains rules with a make-like
syntax:
@verbatim
target1 target2 ... : prerequisite1 prerequisite2 ...
shell script
that builds
the targets
@endverbatim
A target is known to be up-to-date if all its prerequisites are. If it
has no known prerequisites yet the file already exits, it is assumed to
be up-to-date. Obsolete targets are rebuilt thanks to the shell script
provided by the rule.
As with redo, remake supports dynamic dependencies in
addition to these static dependencies. Whenever a script executes
remake prerequisite4 prerequisite5 ..., these prerequisites are
rebuilt if they are obsolete. (So remake acts like
redo-ifchange.) Moreover, all the dependencies are stored in file
.remake so that they are remembered in subsequent runs. Note that
dynamic dependencies from previous runs are only used to decide whether a
target is obsolete; they are not automatically rebuilt when they are
obsolete yet a target depends on them. They will only be rebuilt once the
dynamic call to remake is executed.
In other words, the following two rules have almost the same behavior.
@verbatim
target1 target2 ... : prerequisite1 prerequisite2 ...
shell script
target1 target2 ... :
remake prerequisite1 prerequisite2 ...
shell script
@endverbatim
(There is a difference if the targets already exist, have never been
built before, and the prerequisites are either younger or obsolete, since
the targets will not be rebuilt in the second case.)
The above usage of dynamic dependencies is hardly useful. Their strength
lies in the fact that they can be computed on the fly:
@verbatim
%.o : %.c
gcc -MMD -MF $@.d -o $@ -c $<
remake -r < $@.d
rm $@.d
%.cmo : %.ml
ocamldep $< | remake -r $@
ocamlc -c $<
after.xml: before.xml rules.xsl
xsltproc --load-trace -o after.xml rules.xsl before.xml 2> deps
remake `sed -n -e "\\,//,! s,^.*URL=\"\\([^\"]*\\).*\$,\\1,p" deps`
rm deps
@endverbatim
Note that the first rule fails if any of the header files included by
a C source file has to be automatically generated. In that case, one
should perform a first call to remake them before calling the
compiler. (Dependencies from several calls to remake are
cumulative, so they will all be remembered the next time.)
\section sec-usage Usage
Usage: remake options targets
Options:
- -d: Echo script commands.
- -f FILE: Read FILE as Remakefile.
- -j[N], --jobs=[N]: Allow N jobs at once;
infinite jobs with no argument.
- -k, --keep-going: Keep going when some targets cannot be made.
- -r: Look up targets from the dependencies on standard input.
- -s, --silent, --quiet: Do not echo targets.
\section sec-syntax Syntax
Lines starting with a space character or a tabulation are assumed to be rule
scripts. They are only allowed after a rule header.
Lines starting with # are considered to be comments and are ignored.
They do interrupt rule scripts though.
Any other line is either a rule header or a variable definition. If such a
line ends with a backslash, the following line break is ignored and the line
extends to the next one.
Rule headers are a nonempty list of names, followed by a colon, followed by
another list of names, possibly empty. Variable definitions are a single
name followed by equal followed by a list of names, possibly empty. Basically,
the syntax of a rule is as follows:
@verbatim
targets : prerequisites
shell script
@endverbatim
List of names are space-separated sequences of names. If a name contains a
space character, it should be put into double quotes. Names can not be any
of the following special characters :$(),=". Again, quotation
should be used. Quotation marks can be escaped by a backslash inside
quoted names.
\subsection sec-variables Variables
Variables can be used to factor lists of targets or prerequisites. They are
expanded as they are encountered during Remakefile parsing.
@verbatim
VAR2 = a
VAR1 = c d
VAR2 += $(VAR1) b
$(VAR2) e :
@endverbatim
Variable assignments can appear instead of prerequisites inside non-generic
rules with no script. They are then expanded inside the corresponding
generic rule.
@verbatim
foo.o: CFLAGS += -DBAR
%.o : %.c
gcc $(CFLAGS) -MMD -MF $@.d -o $@ -c $<
remake -r < $@.d
rm $@.d
@endverbatim
Note: contrarily to make, variable names have to be enclosed in
parentheses. For instance, $y is not a shorthand for \$(y) and
is left unexpanded.
\subsection sec-autovars Automatic variables
The following special symbols can appear inside scripts:
- $< expands to the first static prerequisite of the rule.
- $^ expands to all the static prerequisites of the rule, including
duplicates if any.
- $\@ expands to the first target of the rule.
- $* expands to the string that matched % in a generic rule.
- $$ expands to a single dollar symbol.
Note: contrarily to make, there are no corresponding variables. For
instance, $^ is not a shorthand for $(^). Another
difference is that $\@ is always the first target, not the one that
triggered the rule.
\subsection sec-functions Built-in functions
remake also supports a few built-in functions inspired from make.
- $(addprefix prefix, list) returns the list obtained
by prepending its first argument to each element of its second argument.
- $(addsuffix suffix, list) returns the list obtained
by appending its first argument to each element of its second argument.
\section sec-semantics Semantics
\subsection src-obsolete When are targets obsolete?
A target is obsolete:
- if there is no file corresponding to the target, or to one of its siblings
in a multi-target rule,
- if any of its dynamic prerequisites from a previous run or any of its static
prerequisites is obsolete,
- if the latest file corresponding to its siblings or itself is older than any
of its dynamic prerequisites or static prerequisites.
In all the other cases, it is assumed to be up-to-date (and so are all its
siblings). Note that the last rule above says "latest" and not "earliest". While
it might cause some obsolete targets to go unnoticed in corner cases, it allows
for the following kind of rules:
@verbatim
config.h stamp-config_h: config.h.in config.status
./config.status config.h
touch stamp-config_h
@endverbatim
A config.status file generally does not update header files (here
config.h) if they would not change. As a consequence, if not for the
stamp-config_h file above, a header would always be considered obsolete
once one of its prerequisites is modified. Note that touching config.h
rather than stamp-config_h would defeat the point of not updating it
in the first place, since the program files would need to be rebuilt.
Once all the static prerequisites of a target have been rebuilt, remake
checks whether the target still needs to be built. If it was obsolete only
because its prerequisites needed to be rebuilt and none of them changed, the
target is assumed to be up-to-date.
\subsection sec-rules How are targets (re)built?
There are two kinds of rules. If any of the targets or prerequisites contains
a % character, the rule is said to be generic. All the
targets of the rule shall then contain a single % character. All the
other rules are said to be specific.
A rule is said to match a given target:
- if it is specific and the target appears inside its target list,
- if it is generic and there is a way to replace the % character
from one of its targets so that it matches the given target.
When remake tries to build a given target, it looks for a specific rule
that matches it. If there is one and its script is nonempty, it uses it to
rebuild the target.
Otherwise, it looks for a generic rule that match the target. If there are
several matching rules, it chooses the one with the shortest pattern (and if
there are several ones, the earliest one). remake then looks for
specific rules that match each target of the generic rule. All the
prerequisites of these specific rules are added to those of the generic rule.
The script of the generic rule is used to build the target.
Example:
@verbatim
t%1 t2%: p1 p%2
commands building t%1 and t2%
t2z: p4
commands building t2z
ty1: p3
# t2x is built by the first rule (which also builds tx1) and its prerequisites are p1, px2
# t2y is built by the first rule (which also builds ty1) and its prerequisites are p1, py2, p3
# t2z is built by the second rule and its prerequisite is p4
@endverbatim
The set of rules from Remakefile is ill-formed:
- if any specific rule matching a target of the generic rule has a nonempty script,
- if any target of the generic rule is matched by a generic rule with a shorter pattern.
\section sec-compilation Compilation
- On Linux, MacOSX, and BSD: g++ -o remake remake.cpp
- On Windows: g++ -o remake.exe remake.cpp -lws2_32
Installing remake is needed only if Remakefile does not
specify the path to the executable for its recursive calls. Thanks to its
single source file, remake can be shipped inside other packages and
built at configuration time.
\section sec-differences Differences with other build systems
Differences with make:
- Dynamic dependencies are supported.
- For rules with multiple targets, the shell script is executed only once
and is assumed to build all the targets. There is no need for
convoluted rules that are robust enough for parallel builds. For generic
rules, this is similar to the behavior of pattern rules from gmake.
- As with redo, only one shell is run when executing a script,
rather than one per script line. Note that the shells are run with
option -e, thus causing them to exit as soon as an error is
encountered.
- The prerequisites of generic rules (known as implicit rules in make lingo)
are not used to decide between several of them. remake does not
select one for which it could satisfy the dependencies.
- Variables and built-in functions are expanded as they are encountered
during Remakefile parsing.
Differences with redo:
- As with make, it is possible to write the following kind of rules
in remake.
@verbatim
Remakefile: Remakefile.in ./config.status
./config.status Remakefile
@endverbatim
- If a target is already built the first time remake runs, it still
uses the static prerequisites of rules mentioning it to check whether it
needs to be rebuilt. It does not assume it to be up-to-date. As with
redo though, if its obsolete status would be due to a dynamic
prerequisite, it will go unnoticed; it should be removed beforehand.
- Multiple targets are supported.
- remake has almost no features: no checksum-based dependencies, no
compatibility with job servers, etc.
\section sec-limitations Limitations
- When the user calls remake, the current working directory should be
the one containing .remake. Rules are understood relatively to this
directory. If a rule script calls remake, the current working
directory should be the same as the one from the original remake.
- Some cases of ill-formed rules are not caught by remake and can
thus lead to unpredictable behaviors.
\section sec-links Links
@see http://cr.yp.to/redo.html for the philosophy of redo and
https://github.com/apenwarr/redo for an implementation and some comprehensive documentation.
\section sec-licensing Licensing
@author Guillaume Melquiond
@version 0.8
@date 2012-2013
@copyright
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
\n
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
\section sec-internals Internals
The parent remake process acts as a server. The other ones have a
REMAKE_SOCKET environment variable that tells them how to contact the
server. They send the content of the REMAKE_JOB_ID environment variable,
so that the server can associate the child targets to the jobs that
spawned them. They then wait for completion and exit with the status
returned by the server. This is handled by #client_mode.
The server calls #load_dependencies and #save_dependencies to serialize
dynamic dependencies from .remake. It loads Remakefile with
#load_rules. It then runs #server_mode, which calls #server_loop.
When building a target, the following sequence of events happens:
- #start calls #find_rule (and #find_generic_rule) to get the rule.
- It then creates a pseudo-client if the rule has static dependencies, or
calls #run_script otherwise. In both cases, a new job is created and its
targets are put into #job_targets.
- #run_script creates a shell process and stores it in #job_pids. It
increases #running_jobs.
- The child process possibly calls remake with a list of targets.
- #accept_client receives a build request from a child process and adds
it to #clients. It also records the new dependencies of the job into
#dependencies. It increases #waiting_jobs.
- #handle_clients uses #get_status to look up the obsoleteness of the
targets.
- Once the targets of a request have been built or one of them has failed,
#handle_clients calls #complete_request and removes the request from
#clients.
- If the build targets come from a pseudo-client, #complete_request calls
#run_script. Otherwise it sends the reply to the corresponding child
process and decreases #waiting_jobs.
- When a child process ends, #server_loop calls #finalize_job, which
removes the process from #job_pids, decreases #running_jobs, and calls
#complete_job.
- #complete_job removes the job from #job_targets and calls #update_status
to change the status of the targets. It also removes the target files in
case of failure.
*/
#ifdef _WIN32
#define WIN32_LEAN_AND_MEAN
#define WINDOWS
#endif
#include
#include
#include
#include