syntax:
@verbatim
-target1 target2 ... : dependency1 dependency2 ...
+target1 target2 ... : prerequisite1 prerequisite2 ...
shell script
that builds
the targets
@endverbatim
-A target is known to be up-to-date if all its dependencies are. If it
-has no known dependencies yet the file already exits, it is assumed to
+A target is known to be up-to-date if all its prerequisites are. If it
+has no known prerequisites yet the file already exits, it is assumed to
be up-to-date. Obsolete targets are rebuilt thanks to the shell script
provided by the rule.
As with <b>redo</b>, <b>remake</b> supports dynamic dependencies in
addition to these static dependencies. Whenever a script executes
-<tt>remake dependency4 dependency5 ...</tt>, these dependencies are
+<tt>remake prerequisite4 prerequisite5 ...</tt>, these prerequisites are
rebuilt if they are obsolete. (So <b>remake</b> acts like
-<b>redo-ifchange</b>.) Moreover, these dependencies are stored in file
+<b>redo-ifchange</b>.) Moreover, all the dependencies are stored in file
<b>.remake</b> so that they are remembered in subsequent runs. Note that
dynamic dependencies from previous runs are only used to decide whether a
target is obsolete; they are not automatically rebuilt when they are
In other words, the following two rules have almost the same behavior.
@verbatim
-target1 target2 ... : dependency1 dependency2 ...
+target1 target2 ... : prerequisite1 prerequisite2 ...
shell script
target1 target2 ... :
- remake dependency1 dependency2 ...
+ remake prerequisite1 prerequisite2 ...
shell script
@endverbatim
(There is a difference if the targets already exist, have never been
-built before, and the dependencies are either younger or obsolete, since
+built before, and the prerequisites are either younger or obsolete, since
the targets will not be rebuilt in the second case.)
The above usage of dynamic dependencies is hardly useful. Their strength
@verbatim
%.o : %.c
- gcc -MMD -MF $1.d -o $1 -c ${1%.o}.c
- remake -r < $1.d
- rm $1.d
+ gcc -MMD -MF $@.d -o $@ -c $<
+ remake -r < $@.d
+ rm $@.d
%.cmo : %.ml
- ocamldep ${1%.cmo}.ml | remake -r $1
- ocamlc -c ${1%.cmo}.ml
+ ocamldep $< | remake -r $@
+ ocamlc -c $<
after.xml: before.xml rules.xsl
xsltproc --load-trace -o after.xml rules.xsl before.xml 2> deps
- remake $(sed -n -e "\\,//,! s,^.*URL=\"\\([^\"]*\\).*\$,\\1,p" deps)
+ remake `sed -n -e "\\,//,! s,^.*URL=\"\\([^\"]*\\).*\$,\\1,p" deps`
rm deps
@endverbatim
Options:
- <tt>-d</tt>: Echo script commands.
-- <tt>-j[N]</tt>, <tt>--jobs=[N]</tt>: Allow N jobs at once; infinite jobs
- with no argument.
+- <tt>-f FILE</tt>: Read <tt>FILE</tt> as <b>Remakefile</b>.
+- <tt>-j[N]</tt>, <tt>--jobs=[N]</tt>: Allow <tt>N</tt> jobs at once;
+ infinite jobs with no argument.
- <tt>-k</tt>, <tt>--keep-going</tt>: Keep going when some targets cannot be made.
- <tt>-r</tt>: Look up targets from the dependencies on standard input.
- <tt>-s</tt>, <tt>--silent</tt>, <tt>--quiet</tt>: Do not echo targets.
Lines starting with <tt>#</tt> are considered to be comments and are ignored.
They do interrupt rule scripts though.
-Any other line is either a rule header or a variable definition. If such a
+Any other line is either a variable definition or a rule header. If such a
line ends with a backslash, the following line break is ignored and the line
extends to the next one.
+Variable definitions are a single name followed by equal followed by a list
+of names, possibly empty.
+
Rule headers are a nonempty list of names, followed by a colon, followed by
-another list of names, possibly empty. Variable definitions are a single
-name followed by equal followed by a list of names, possibly empty. Basically,
-the syntax of a rule is as follows:
+another list of names, possibly empty. Basically, the syntax of a rule is as
+follows:
@verbatim
targets : prerequisites
\subsection sec-variables Variables
-Variables can be used to factor lists of targets or dependencies. They are
+Variables can be used to factor lists of targets or prerequisites. They are
expanded as they are encountered during <b>Remakefile</b> parsing.
@verbatim
+VAR2 = a
VAR1 = c d
-VAR2 = a $(VAR1) b
+VAR2 += $(VAR1) b
$(VAR2) e :
@endverbatim
-Variables can be used inside rule scripts; they are available as non-exported
-shell variables there.
+Variable assignments can appear instead of prerequisites inside non-generic
+rules with no script. They are then expanded inside the corresponding
+generic rule.
+
+@verbatim
+foo.o: CFLAGS += -DBAR
+
+%.o : %.c
+ gcc $(CFLAGS) -MMD -MF $@.d -o $@ -c $<
+ remake -r < $@.d
+ rm $@.d
+@endverbatim
+
+Note: contrarily to <b>make</b>, variable names have to be enclosed in
+parentheses. For instance, <tt>$y</tt> is not a shorthand for <tt>\$(y)</tt> and
+is left unexpanded.
+
+\subsection sec-autovars Automatic variables
+
+The following special symbols can appear inside scripts:
+
+- <tt>$<</tt> expands to the first static prerequisite of the rule.
+- <tt>$^</tt> expands to all the static prerequisites of the rule, including
+ duplicates if any.
+- <tt>$\@</tt> expands to the first target of the rule.
+- <tt>$*</tt> expands to the string that matched <tt>%</tt> in a generic rule.
+- <tt>$$</tt> expands to a single dollar symbol.
+
+Note: contrarily to <b>make</b>, there are no corresponding variables. For
+instance, <tt>$^</tt> is not a shorthand for <tt>$(^)</tt>. Another
+difference is that <tt>$\@</tt> is always the first target, not the one that
+triggered the rule.
\subsection sec-functions Built-in functions
- <tt>$(addsuffix <i>suffix</i>, <i>list</i>)</tt> returns the list obtained
by appending its first argument to each element of its second argument.
-Note that functions are ignored inside scripts.
+\subsection sec-order Order-only prerequisites
+
+If the static prerequisites of a rule contain a pipe symbol, prerequisites
+on its right do not cause the targets to become obsolete if they are newer
+(unless they are also dynamically registered as dependencies). They are
+meant to be used when the targets do not directly depend on them, but the
+computation of their dynamic dependencies does.
+
+@verbatim
+%.o : %.c | parser.h
+ gcc -MMD -MF $@.d -o $@ -c $<
+ remake -r < $@.d
+ rm $@.d
+
+parser.c parser.h: parser.y
+ yacc -d -o parser.c parser.y
+@endverbatim
+
+\subsection sec-special-var Special variables
+
+Variable <tt>.OPTIONS</tt> is handled specially. Its content enables some
+features of <b>remake</b> that are not enabled by default.
+
+- <tt>variable-propagation</tt>: When a variable is set in the prerequisite
+ part of a rule, it is propagated to the rules of all the targets this rule
+ depends on. This option also enables variables to be set on the command
+ line. Note that, as in <b>make</b>, this features introduces non-determinism:
+ the content of some variables will depend on the build order.
\section sec-semantics Semantics
- if there is no file corresponding to the target, or to one of its siblings
in a multi-target rule,
-- if any of its dynamic dependencies from a previous run or any of its static
+- if any of its dynamic prerequisites from a previous run or any of its static
prerequisites is obsolete,
- if the latest file corresponding to its siblings or itself is older than any
- of its dynamic dependencies or static prerequisites.
+ of its dynamic prerequisites or static prerequisites.
In all the other cases, it is assumed to be up-to-date (and so are all its
siblings). Note that the last rule above says "latest" and not "earliest". While
in the first place, since the program files would need to be rebuilt.
Once all the static prerequisites of a target have been rebuilt, <b>remake</b>
-checks if the target still needs to be built. If it was obsolete only because
-its dependencies needed to be rebuilt and none of them changed, the target is
-assumed to be up-to-date.
+checks whether the target still needs to be built. If it was obsolete only
+because its prerequisites needed to be rebuilt and none of them changed, the
+target is assumed to be up-to-date.
\subsection sec-rules How are targets (re)built?
that matches it. If there is one and its script is nonempty, it uses it to
rebuild the target.
-Otherwise, it looks for a generic rule that match the target. If there are
+Otherwise, it looks for a generic rule that matches the target. If there are
several matching rules, it chooses the one with the shortest pattern (and if
there are several ones, the earliest one). <b>remake</b> then looks for
specific rules that match each target of the generic rule. All the
rather than one per script line. Note that the shells are run with
option <tt>-e</tt>, thus causing them to exit as soon as an error is
encountered.
-- The dependencies of generic rules (known as implicit rules in make lingo)
+- The prerequisites of generic rules (known as implicit rules in make lingo)
are not used to decide between several of them. <b>remake</b> does not
select one for which it could satisfy the dependencies.
- Variables and built-in functions are expanded as they are encountered
during <b>Remakefile</b> parsing.
+- Target-specific variables are not propagated, unless specifically enabled,
+ since this causes non-deterministic builds. This is the same for variables
+ set on the command line.
Differences with <b>redo</b>:
uses the static prerequisites of rules mentioning it to check whether it
needs to be rebuilt. It does not assume it to be up-to-date. As with
<b>redo</b> though, if its obsolete status would be due to a dynamic
- dependency, it will go unnoticed; it should be removed beforehand.
-- <b>remake</b> has almost no features: no checksum-based dependencies, no
- compatibility with token servers, etc.
-
-Differences with both <b>make</b> and <b>redo</b>:
-
+ prerequisite, it will go unnoticed; it should be removed beforehand.
- Multiple targets are supported.
-- When executing shell scripts, positional variables <tt>$1</tt>,
- <tt>$2</tt>, etc, point to the target names of the rule obtained after
- substituting <tt>%</tt>. No other variables are defined.
+- <b>remake</b> has almost no features: no checksum-based dependencies, no
+ compatibility with job servers, etc.
\section sec-limitations Limitations
-- When the user or a script calls <b>remake</b>, the current working
- directory should be the one containing <b>Remakefile</b> (and thus
- <b>.remake</b> too).
+- If a rule script calls <b>remake</b>, the current working directory should
+ be the directory containing <b>Remakefile</b> (or the working directory
+ from the original <b>remake</b> if it was called with option <b>-f</b>).
+- As with <b>make</b>, variables passed on the command line should keep
+ the same values, to ensure deterministic builds.
- Some cases of ill-formed rules are not caught by <b>remake</b> and can
thus lead to unpredictable behaviors.
\section sec-licensing Licensing
@author Guillaume Melquiond
-@version 0.5
+@version 0.10
@date 2012-2013
@copyright
This program is free software: you can redistribute it and/or modify
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
+
+\section sec-internals Internals
+
+The parent <b>remake</b> process acts as a server. The other ones have a
+REMAKE_SOCKET environment variable that tells them how to contact the
+server. They send the content of the REMAKE_JOB_ID environment variable,
+so that the server can associate the child targets to the jobs that
+spawned them. They then wait for completion and exit with the status
+returned by the server. This is handled by #client_mode.
+
+The server calls #load_dependencies and #save_dependencies to serialize
+dynamic dependencies from <b>.remake</b>. It loads <b>Remakefile</b> with
+#load_rules. It then runs #server_mode, which calls #server_loop.
+
+When building a target, the following sequence of events happens:
+
+- #start calls #find_rule (and #find_generic_rule) to get the rule.
+- It then creates a pseudo-client if the rule has static dependencies, or
+ calls #run_script otherwise. In both cases, a new job is created; the
+ rule and the variables are stored into #jobs.
+- #run_script creates a shell process and stores it in #job_pids. It
+ increases #running_jobs.
+- The child process possibly calls <b>remake</b> with a list of targets.
+- #accept_client receives a build request from a child process and adds
+ it to #clients. It also records the new dependencies of the job into
+ #dependencies. It increases #waiting_jobs.
+- #handle_clients uses #get_status to look up the obsoleteness of the
+ targets.
+- Once the targets of a request have been built or one of them has failed,
+ #handle_clients calls #complete_request and removes the request from
+ #clients.
+- If the build targets come from a pseudo-client, #complete_request calls
+ #run_script. Otherwise it sends the reply to the corresponding child
+ process and decreases #waiting_jobs.
+- When a child process ends, #server_loop calls #finalize_job, which
+ removes the process from #job_pids, decreases #running_jobs, and calls
+ #complete_job.
+- #complete_job removes the job from #jobs and calls #update_status
+ to change the status of the targets. It also removes the target files in
+ case of failure.
*/
#ifdef _WIN32
#include <vector>
#include <cassert>
#include <cstdlib>
+#include <cstring>
#include <ctime>
#include <errno.h>
#include <fcntl.h>
#include <sys/wait.h>
typedef int socket_t;
enum { INVALID_SOCKET = -1 };
+extern char **environ;
#endif
#if defined(WINDOWS) || defined(MACOSX)
typedef std::map<std::string, status_t> status_map;
+/**
+ * Delayed assignment to a variable.
+ */
+struct assign_t
+{
+ bool append;
+ string_list value;
+};
+
+typedef std::map<std::string, assign_t> assign_map;
+
/**
* A rule loaded from Remakefile.
*/
struct rule_t
{
string_list targets; ///< Files produced by this rule.
- string_list deps; ///< Files used for an implicit call to remake at the start of the script.
+ string_list deps; ///< Dependencies used for an implicit call to remake at the start of the script.
+ string_list wdeps; ///< Like #deps, except that they are not registered as dependencies.
+ assign_map assigns; ///< Assignment of variables.
std::string script; ///< Shell script for building the targets.
};
typedef std::map<std::string, ref_ptr<rule_t> > rule_map;
-typedef std::map<int, string_list> job_targets_map;
+/**
+ * A job created from a set of rules.
+ */
-typedef std::map<int, variable_map> job_variables_map;
+struct job_t
+{
+ rule_t rule; ///< Original rule.
+ std::string stem; ///< Pattern used to instantiate the generic rule, if any.
+ variable_map vars; ///< Values of local variables.
+};
+
+typedef std::map<int, job_t> job_map;
typedef std::map<pid_t, int> pid_job_map;
/**
- * Client waiting for a request complete.
+ * Client waiting for a request to complete.
*
* There are two kinds of clients:
* - real clients, which are instances of remake created by built scripts,
bool failed; ///< Whether some targets failed in mode -k.
string_list pending; ///< Targets not yet started.
string_set running; ///< Targets being built.
- rule_t *delayed; ///< Rule that implicitly created a dependency client, and which script has to be started on request completion.
- variable_map variables; ///< Variable that are passed on the command line of a remake invocation.
- client_t(): socket(INVALID_SOCKET), job_id(-1), failed(false), delayed(NULL) {}
+ variable_map vars; ///< Variables set on request.
+ bool delayed; ///< Whether it is a dependency client and a script has to be started on request completion.
+ client_t(): socket(INVALID_SOCKET), job_id(-1), failed(false), delayed(false) {}
};
typedef std::list<client_t> client_list;
/**
* Map from variable names to their content.
+ * Initialized with the values passed on the command line.
*/
static variable_map variables;
-/**
- * Precomputed variable assignments for shell usage.
- */
-static std::string variable_block;
-
/**
* Map from targets to their known dependencies.
*/
static rule_map specific_rules;
/**
- * Map from jobs to targets being built.
+ * Map of jobs being built.
*/
-static job_targets_map job_targets;
-
-/**
- * Map from jobs to job specific variables
- */
-static job_variables_map job_variables;
+static job_map jobs;
/**
* Map from jobs to shell pids.
/**
* Global counter used to produce increasing job numbers.
- * @see job_targets
+ * @see jobs
*/
static int job_counter = 0;
*/
static bool build_failure;
+#ifndef WINDOWS
/**
* Name of the server socket in the file system.
*/
static char *socket_name;
+#endif
/**
* Name of the first target of the first specific rule, used for default run.
*/
static bool echo_scripts = false;
+/**
+ * Time at the start of the program.
+ */
static time_t now = time(NULL);
+/**
+ * Directory with respect to which command-line names are relative.
+ */
static std::string working_dir;
+/**
+ * Directory with respect to which targets are relative.
+ */
+static std::string prefix_dir;
+
+/**
+ * Whether the prefix directory is different from #working_dir.
+ */
+static bool changed_prefix_dir;
+
+/**
+ * Whether target-specific variables are propagated to prerequisites.
+ */
+static bool propagate_vars = false;
+
#ifndef WINDOWS
static volatile sig_atomic_t got_SIGCHLD = 0;
-static void child_sig_handler(int)
+static void sigchld_handler(int)
{
got_SIGCHLD = 1;
}
+
+static void sigint_handler(int)
+{
+ // Child processes will receive the signal too, so just prevent
+ // new jobs from starting and wait for the running jobs to fail.
+ keep_going = false;
+}
#endif
struct log
#define DEBUG_close if ((auto_close.still_open = false), debug.active) debug(false)
/**
- * Return the original string if it does not contain any special characters,
- * a quoted and escaped string otherwise.
+ * Strong typedef for strings that need escaping.
+ * @note The string is stored as a reference, so the constructed object is
+ * meant to be immediately consumed.
*/
-static std::string escape_string(std::string const &s)
+struct escape_string
{
+ std::string const &input;
+ escape_string(std::string const &s): input(s) {}
+};
+
+/**
+ * Write the string in @a se to @a out if it does not contain any special
+ * characters, a quoted and escaped string otherwise.
+ */
+static std::ostream &operator<<(std::ostream &out, escape_string const &se)
+{
+ std::string const &s = se.input;
char const *quoted_char = ",: '";
char const *escaped_char = "\"\\$!";
bool need_quotes = false;
- size_t len = s.length(), nb = len;
+ char *buf = NULL;
+ size_t len = s.length(), last = 0, j = 0;
for (size_t i = 0; i < len; ++i)
{
- if (strchr(quoted_char, s[i])) need_quotes = true;
- if (strchr(escaped_char, s[i])) ++nb;
- }
- if (nb != len) need_quotes = true;
- if (!need_quotes) return s;
- std::string t(nb + 2, '\\');
- t[0] = '"';
- for (size_t i = 0, j = 1; i < len; ++i, ++j)
- {
- if (strchr(escaped_char, s[i])) ++j;
- t[j] = s[i];
- }
- t[nb + 1] = '"';
- return t;
+ if (strchr(escaped_char, s[i]))
+ {
+ need_quotes = true;
+ if (!buf) buf = new char[len * 2];
+ memcpy(&buf[j], &s[last], i - last);
+ j += i - last;
+ buf[j++] = '\\';
+ buf[j++] = s[i];
+ last = i + 1;
+ }
+ if (!need_quotes && strchr(quoted_char, s[i]))
+ need_quotes = true;
+ }
+ if (!need_quotes) return out << s;
+ out << '"';
+ if (!buf) return out << s << '"';
+ out.write(buf, j);
+ out.write(&s[last], len - last);
+ delete[] buf;
+ return out << '"';
}
+/**
+ * @defgroup paths Path helpers
+ *
+ * @{
+ */
+
/**
* Initialize #working_dir.
*/
-void init_working_dir(const char* argv0)
+static void init_working_dir()
{
char buf[1024];
-#ifdef WINDOWS
- char const *delim = "/\\";
-#else
- char delim = '/';
-#endif
- if (!getenv("REMAKE_SOCKET"))
- {
- std::string path = argv0;
-
- size_t found = path.find_last_of(delim);
- if (found != std::string::npos)
- {
- path = path.substr(0, found);
- std::cout << "Entering directory `" << path << "'" << std::endl;
- if (chdir(path.c_str()))
- {
- perror("Failed to change working directory");
- exit(EXIT_FAILURE);
- }
- }
- }
-
char *res = getcwd(buf, sizeof(buf));
if (!res)
{
exit(EXIT_FAILURE);
}
working_dir = buf;
+#ifdef WINDOWS
+ for (size_t i = 0, l = working_dir.size(); i != l; ++i)
+ {
+ if (working_dir[i] == '\\') working_dir[i] = '/';
+ }
+#endif
+ prefix_dir = working_dir;
}
/**
- * Normalize an absolute path with respect to the working directory.
- * Paths outside the working subtree are left unchanged.
+ * Initialize #prefix_dir and switch to it.
*/
-static std::string normalize_abs(std::string const &s)
+static void init_prefix_dir()
{
- size_t l = working_dir.length();
- if (s.compare(0, l, working_dir)) return s;
+ for (;;)
+ {
+ struct stat s;
+ if (stat((prefix_dir + "/Remakefile").c_str(), &s) == 0)
+ {
+ if (!changed_prefix_dir) return;
+ if (chdir(prefix_dir.c_str()))
+ {
+ perror("Failed to change working directory");
+ exit(EXIT_FAILURE);
+ }
+ if (show_targets)
+ {
+ std::cout << "remake: Entering directory `" << prefix_dir << '\'' << std::endl;
+ }
+ return;
+ }
+ size_t pos = prefix_dir.find_last_of('/');
+ if (pos == std::string::npos)
+ {
+ std::cerr << "Failed to locate Remakefile in the current directory or one of its parents" << std::endl;
+ exit(EXIT_FAILURE);
+ }
+ prefix_dir.erase(pos);
+ changed_prefix_dir = true;
+ }
+}
+
+/**
+ * Normalize an absolute path with respect to @a p.
+ * Paths outside the subtree are left unchanged.
+ */
+static std::string normalize_abs(std::string const &s, std::string const &p)
+{
+ size_t l = p.length();
+ if (s.compare(0, l, p)) return s;
size_t ll = s.length();
if (ll == l) return ".";
if (s[l] != '/')
}
/**
- * Normalize a target name.
+ * Normalize path @a s (possibly relative to @a w) with respect to @a p.
+ *
+ * - If both @a p and @a w are empty, the function just removes ".", "..", "//".
+ * - If only @a p is empty, the function returns an absolute path.
*/
-static std::string normalize(std::string const &s)
+static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
{
#ifdef WINDOWS
char const *delim = "/\\";
#else
char delim = '/';
#endif
- size_t prev = 0, len = s.length();
size_t pos = s.find_first_of(delim);
- if (pos == std::string::npos) return s;
+ if (pos == std::string::npos && w == p) return s;
bool absolute = pos == 0;
+ if (!absolute && w != p && !w.empty())
+ return normalize(w + '/' + s, w, p);
+ size_t prev = 0, len = s.length();
string_list l;
for (;;)
{
if (n == "..")
{
if (!l.empty()) l.pop_back();
- else if (!absolute)
- return normalize(working_dir + '/' + s);
+ else if (!absolute && !w.empty())
+ return normalize(w + '/' + s, w, p);
}
else if (n != ".")
l.push_back(n);
n.push_back('/');
n.append(*i);
}
- if (absolute) return normalize_abs(n);
+ if (absolute && !p.empty()) return normalize_abs(n, p);
return n;
}
/**
* Normalize the content of a list of targets.
*/
-static void normalize_list(string_list &l)
+static void normalize_list(string_list &l, std::string const &w, std::string const &p)
{
for (string_list::iterator i = l.begin(),
i_end = l.end(); i != i_end; ++i)
{
- *i = normalize(*i);
+ *i = normalize(*i, w, p);
}
}
+/** @} */
+
+/**
+ * @defgroup lexer Lexer
+ *
+ * @{
+ */
+
/**
* Skip spaces.
*/
}
/**
- * Skip end of line.
+ * Skip empty lines.
*/
-static void skip_eol(std::istream &in)
+static void skip_empty(std::istream &in)
{
char c;
while (strchr("\r\n", (c = in.get()))) {}
if (in.good()) in.putback(c);
}
-enum token_e { Word, Eol, Eof, Colon, Equal, Dollar, Rightpar, Comma };
+/**
+ * Skip end of line. If @a multi is true, skip the following empty lines too.
+ * @return true if there was a line to end.
+ */
+static bool skip_eol(std::istream &in, bool multi = false)
+{
+ char c = in.get();
+ if (c == '\r') c = in.get();
+ if (c != '\n' && in.good()) in.putback(c);
+ if (c != '\n' && !in.eof()) return false;
+ if (multi) skip_empty(in);
+ return true;
+}
+
+enum
+{
+ Unexpected = 0,
+ Word = 1 << 1,
+ Colon = 1 << 2,
+ Equal = 1 << 3,
+ Dollarpar = 1 << 4,
+ Rightpar = 1 << 5,
+ Comma = 1 << 6,
+ Plusequal = 1 << 7,
+ Pipe = 1 << 8,
+};
/**
- * Skip spaces and return the kind of the next token.
+ * Skip spaces and peek at the next token.
+ * If it is one of @a mask, skip it (if it is not Word) and return it.
+ * @note For composite tokens allowed by @a mask, input characters might
+ * have been eaten even for an Unexpected result.
*/
-static token_e next_token(std::istream &in)
+static int expect_token(std::istream &in, int mask)
{
while (true)
{
skip_spaces(in);
char c = in.peek();
- if (!in.good()) return Eof;
+ if (!in.good()) return Unexpected;
+ int tok;
switch (c)
{
- case ':': return Colon;
- case ',': return Comma;
- case '=': return Equal;
- case '$': return Dollar;
- case ')': return Rightpar;
case '\r':
- case '\n':
- return Eol;
- case '\\':
+ case '\n': return Unexpected;
+ case ':': tok = Colon; break;
+ case ',': tok = Comma; break;
+ case '=': tok = Equal; break;
+ case ')': tok = Rightpar; break;
+ case '|': tok = Pipe; break;
+ case '$':
+ if (!(mask & Dollarpar)) return Unexpected;
in.ignore(1);
- c = in.peek();
- if (c != '\r' && c != '\n')
- {
- in.putback('\\');
- return Word;
- }
- skip_eol(in);
+ tok = Dollarpar;
+ if (in.peek() != '(') return Unexpected;
break;
+ case '+':
+ if (!(mask & Plusequal)) return Unexpected;
+ in.ignore(1);
+ tok = Plusequal;
+ if (in.peek() != '=') return Unexpected;
+ break;
+ case '\\':
+ in.ignore(1);
+ if (skip_eol(in)) continue;
+ in.putback('\\');
+ return mask & Word ? Word : Unexpected;
default:
- return Word;
+ return mask & Word ? Word : Unexpected;
}
+ if (!(tok & mask)) return Unexpected;
+ in.ignore(1);
+ return tok;
}
}
/**
* Read a (possibly quoted) word.
*/
-static std::string read_word(std::istream &in)
+static std::string read_word(std::istream &in, bool detect_equal = true)
{
- int c = in.get();
+ int c = in.peek();
std::string res;
if (!in.good()) return res;
- char const *separators = " \t\r\n:$(),=\"";
+ char const *separators = " \t\r\n$(),:";
bool quoted = c == '"';
- if (!quoted)
- {
- if (strchr(separators, c))
- {
- in.putback(c);
- return res;
- }
- res += c;
- }
+ if (quoted) in.ignore(1);
+ bool plus = false;
while (true)
{
- c = in.get();
+ c = in.peek();
if (!in.good()) return res;
if (quoted)
{
+ in.ignore(1);
if (c == '\\')
res += in.get();
else if (c == '"')
- return res;
+ quoted = false;
else
res += c;
+ continue;
}
- else
+ if (detect_equal && c == '=')
{
- if (strchr(separators, c))
- {
- in.putback(c);
- return res;
- }
- res += c;
+ if (plus) in.putback('+');
+ return res;
}
+ if (plus)
+ {
+ res += '+';
+ plus = false;
+ }
+ if (strchr(separators, c)) return res;
+ in.ignore(1);
+ if (detect_equal && c == '+') plus = true;
+ else res += c;
}
}
-static string_list read_words(std::istream &in);
+/** @} */
+
+/**
+ * @defgroup stream Token streams
+ *
+ * @{
+ */
/**
- * Execute a built-in function @a name and append its result to @a dest.
+ * Possible results from word producers.
*/
-static void execute_function(std::istream &in, std::string const &name, string_list &dest)
+enum input_status
{
- if (false)
+ Success,
+ SyntaxError,
+ Eof
+};
+
+/**
+ * Interface for word producers.
+ */
+struct generator
+{
+ virtual ~generator() {}
+ virtual input_status next(std::string &) = 0;
+};
+
+/**
+ * Generator for the words of a variable.
+ */
+struct variable_generator: generator
+{
+ std::string name;
+ string_list::const_iterator vcur, vend;
+ variable_generator(std::string const &, variable_map const *);
+ input_status next(std::string &);
+};
+
+variable_generator::variable_generator(std::string const &n,
+ variable_map const *local_variables): name(n)
+{
+ if (local_variables)
{
- error:
- std::cerr << "Failed to load rules: syntax error" << std::endl;
- exit(EXIT_FAILURE);
- }
- skip_spaces(in);
- string_list fix = read_words(in);
- if (next_token(in) != Comma) goto error;
- in.ignore(1);
- string_list names = read_words(in);
- if (next_token(in) != Rightpar) goto error;
- in.ignore(1);
- size_t fixl = fix.size();
- if (name == "addprefix")
- {
- for (string_list::const_iterator i = names.begin(),
- i_end = names.end(); i != i_end; ++i)
+ variable_map::const_iterator i = local_variables->find(name);
+ if (i != local_variables->end())
{
- if (!fixl)
- {
- dest.push_back(*i);
- continue;
- }
- string_list::const_iterator k = fix.begin();
- for (size_t j = 1; j != fixl; ++j)
- {
- dest.push_back(*k++);
- }
- dest.push_back(*k++ + *i);
+ vcur = i->second.begin();
+ vend = i->second.end();
+ return;
}
}
- else if (name == "addsuffix")
+ variable_map::const_iterator i = variables.find(name);
+ if (i == variables.end()) return;
+ vcur = i->second.begin();
+ vend = i->second.end();
+}
+
+input_status variable_generator::next(std::string &res)
+{
+ if (vcur != vend)
{
- for (string_list::const_iterator i = names.begin(),
- i_end = names.end(); i != i_end; ++i)
+ res = *vcur;
+ ++vcur;
+ return Success;
+ }
+ return Eof;
+}
+
+/**
+ * Generator for the words of an input stream.
+ */
+struct input_generator
+{
+ std::istream ∈
+ generator *nested;
+ variable_map const *local_variables;
+ bool earliest_exit, done;
+ input_generator(std::istream &i, variable_map const *lv, bool e = false)
+ : in(i), nested(NULL), local_variables(lv), earliest_exit(e), done(false) {}
+ input_status next(std::string &);
+ ~input_generator() { assert(!nested); }
+};
+
+static generator *get_function(input_generator const &, std::string const &);
+
+input_status input_generator::next(std::string &res)
+{
+ if (nested)
+ {
+ restart:
+ input_status s = nested->next(res);
+ if (s == Success) return Success;
+ delete nested;
+ nested = NULL;
+ if (s == SyntaxError) return SyntaxError;
+ }
+ if (done) return Eof;
+ if (earliest_exit) done = true;
+ switch (expect_token(in, Word | Dollarpar))
+ {
+ case Word:
+ res = read_word(in, false);
+ return Success;
+ case Dollarpar:
+ {
+ std::string name = read_word(in, false);
+ if (name.empty()) return SyntaxError;
+ if (expect_token(in, Rightpar))
+ nested = new variable_generator(name, local_variables);
+ else
{
- if (!fixl)
- {
- dest.push_back(*i);
- continue;
- }
- string_list::const_iterator k = fix.begin();
- dest.push_back(*i + *k++);
- for (size_t j = 1; j != fixl; ++j)
- {
- dest.push_back(*k++);
- }
+ nested = get_function(*this, name);
+ if (!nested) return SyntaxError;
}
+ goto restart;
+ }
+ default:
+ return Eof;
}
- else goto error;
}
/**
- * Read a list of words, possibly executing functions.
+ * Read a list of words from an input generator.
+ * @return false if a syntax error was encountered.
*/
-static string_list read_words(std::istream &in)
+static bool read_words(input_generator &in, string_list &res)
{
- if (false)
+ while (true)
{
- error:
- std::cerr << "Failed to load rules: syntax error" << std::endl;
- exit(EXIT_FAILURE);
+ res.push_back(std::string());
+ input_status s = in.next(res.back());
+ if (s == Success) continue;
+ res.pop_back();
+ return s == Eof;
}
- string_list res;
- while (true)
+}
+
+static bool read_words(std::istream &in, string_list &res)
+{
+ input_generator gen(in, NULL);
+ return read_words(gen, res);
+}
+
+/**
+ * Generator for the result of function addprefix.
+ */
+struct addprefix_generator: generator
+{
+ input_generator gen;
+ string_list pre;
+ string_list::const_iterator prei;
+ size_t prej, prel;
+ std::string suf;
+ addprefix_generator(input_generator const &, bool &);
+ input_status next(std::string &);
+};
+
+addprefix_generator::addprefix_generator(input_generator const &top, bool &ok)
+ : gen(top.in, top.local_variables)
+{
+ if (!read_words(gen, pre)) return;
+ if (!expect_token(gen.in, Comma)) return;
+ prej = 0;
+ prel = pre.size();
+ ok = true;
+}
+
+input_status addprefix_generator::next(std::string &res)
+{
+ if (prej)
{
- switch (next_token(in))
+ produce:
+ if (prej == prel)
{
- case Word:
- res.push_back(read_word(in));
- break;
- case Dollar:
+ res = *prei + suf;
+ prej = 0;
+ }
+ else
{
- in.ignore(1);
- if (in.get() != '(') goto error;
- std::string name = read_word(in);
- if (name.empty()) goto error;
- token_e tok = next_token(in);
- if (tok == Rightpar)
- {
- in.ignore(1);
- variable_map::const_iterator i = variables.find(name);
- if (i != variables.end())
- res.insert(res.end(), i->second.begin(), i->second.end());
- }
- else execute_function(in, name, res);
- break;
+ res = *prei++;
+ ++prej;
}
- default:
- return res;
+ return Success;
+ }
+ switch (gen.next(res))
+ {
+ case Success:
+ if (!prel) return Success;
+ prei = pre.begin();
+ prej = 1;
+ suf = res;
+ goto produce;
+ case Eof:
+ return expect_token(gen.in, Rightpar) ? Eof : SyntaxError;
+ default:
+ return SyntaxError;
+ }
+}
+
+/**
+ * Generator for the result of function addsuffix.
+ */
+struct addsuffix_generator: generator
+{
+ input_generator gen;
+ string_list suf;
+ string_list::const_iterator sufi;
+ size_t sufj, sufl;
+ std::string pre;
+ addsuffix_generator(input_generator const &, bool &);
+ input_status next(std::string &);
+};
+
+addsuffix_generator::addsuffix_generator(input_generator const &top, bool &ok)
+ : gen(top.in, top.local_variables)
+{
+ if (!read_words(gen, suf)) return;
+ if (!expect_token(gen.in, Comma)) return;
+ sufj = 0;
+ sufl = suf.size();
+ ok = true;
+}
+
+input_status addsuffix_generator::next(std::string &res)
+{
+ if (sufj)
+ {
+ if (sufj != sufl)
+ {
+ res = *sufi++;
+ ++sufj;
+ return Success;
}
+ sufj = 0;
+ }
+ switch (gen.next(res))
+ {
+ case Success:
+ if (!sufl) return Success;
+ sufi = suf.begin();
+ sufj = 1;
+ res += *sufi++;
+ return Success;
+ case Eof:
+ return expect_token(gen.in, Rightpar) ? Eof : SyntaxError;
+ default:
+ return SyntaxError;
}
}
/**
- * Serialize a variable map.
- */
-std::string serialize_variables(variable_map &vars, char sep = ' ')
-{
- std::ostringstream buf;
- for(variable_map::const_iterator i = vars.begin(),
- i_end = vars.end(); i != i_end; ++i)
- {
- buf << i->first << '=';
- bool first = true;
- std::string val;
- for (string_list::const_iterator j = i->second.begin(),
- j_end = i->second.end(); j != j_end; ++j)
- {
- if (first) first = false;
- else val += " ";
- val += *j;
- }
- buf << escape_string(val) << sep;
- }
- std::string s = buf.str();
- return s;
+ * Return a generator for function @a name.
+ */
+static generator *get_function(input_generator const &in, std::string const &name)
+{
+ skip_spaces(in.in);
+ generator *g = NULL;
+ bool ok = false;
+ if (name == "addprefix") g = new addprefix_generator(in, ok);
+ else if (name == "addsuffix") g = new addsuffix_generator(in, ok);
+ if (!g || ok) return g;
+ delete g;
+ return NULL;
}
+/** @} */
+
+/**
+ * @defgroup database Dependency database
+ *
+ * @{
+ */
+
/**
* Load dependencies from @a in.
*/
static void load_dependencies(std::istream &in)
{
+ if (false)
+ {
+ error:
+ std::cerr << "Failed to load database" << std::endl;
+ exit(EXIT_FAILURE);
+ }
+
while (!in.eof())
{
- string_list targets = read_words(in);
- if (targets.empty()) return;
+ string_list targets;
+ if (!read_words(in, targets)) goto error;
+ if (in.eof()) return;
+ if (targets.empty()) goto error;
DEBUG << "reading dependencies of target " << targets.front() << std::endl;
- if (in.get() != ':')
- {
- std::cerr << "Failed to load database" << std::endl;
- exit(EXIT_FAILURE);
- }
+ if (in.get() != ':') goto error;
ref_ptr<dependency_t> dep;
dep->targets = targets;
- string_list d = read_words(in);
- dep->deps.insert(d.begin(), d.end());
+ string_list deps;
+ if (!read_words(in, deps)) goto error;
+ dep->deps.insert(deps.begin(), deps.end());
for (string_list::const_iterator i = targets.begin(),
i_end = targets.end(); i != i_end; ++i)
{
dependencies[*i] = dep;
}
- skip_eol(in);
+ skip_empty(in);
}
}
std::ifstream in(".remake");
if (!in.good())
{
- DEBUG_close << "not found\n";
- return;
+ DEBUG_close << "not found\n";
+ return;
+ }
+ load_dependencies(in);
+}
+
+
+/**
+ * Save all the dependencies in file <tt>.remake</tt>.
+ */
+static void save_dependencies()
+{
+ DEBUG_open << "Saving database... ";
+ std::ofstream db(".remake");
+ while (!dependencies.empty())
+ {
+ ref_ptr<dependency_t> dep = dependencies.begin()->second;
+ for (string_list::const_iterator i = dep->targets.begin(),
+ i_end = dep->targets.end(); i != i_end; ++i)
+ {
+ db << escape_string(*i) << ' ';
+ dependencies.erase(*i);
+ }
+ db << ':';
+ for (string_set::const_iterator i = dep->deps.begin(),
+ i_end = dep->deps.end(); i != i_end; ++i)
+ {
+ db << ' ' << escape_string(*i);
+ }
+ db << std::endl;
+ }
+}
+
+/** @} */
+
+static void merge_rule(rule_t &dest, rule_t const &src);
+
+/**
+ * @defgroup parser Rule parser
+ *
+ * @{
+ */
+
+/**
+ * Register a specific rule with an empty script:
+ *
+ * - Check that none of the targets already has an associated rule with a
+ * nonempty script.
+ * - Create a new rule with a single target for each target, if needed.
+ * - Add the prerequisites of @a rule to all these associated rules.
+ */
+static void register_transparent_rule(rule_t const &rule, string_list const &targets)
+{
+ assert(rule.script.empty());
+ for (string_list::const_iterator i = targets.begin(),
+ i_end = targets.end(); i != i_end; ++i)
+ {
+ std::pair<rule_map::iterator, bool> j =
+ specific_rules.insert(std::make_pair(*i, ref_ptr<rule_t>()));
+ ref_ptr<rule_t> &r = j.first->second;
+ if (j.second)
+ {
+ r = ref_ptr<rule_t>(rule);
+ r->targets = string_list(1, *i);
+ continue;
+ }
+ if (!r->script.empty())
+ {
+ std::cerr << "Failed to load rules: " << *i
+ << " cannot be the target of several rules" << std::endl;
+ exit(EXIT_FAILURE);
+ }
+ assert(r->targets.size() == 1 && r->targets.front() == *i);
+ merge_rule(*r, rule);
+ }
+
+ for (string_list::const_iterator i = targets.begin(),
+ i_end = targets.end(); i != i_end; ++i)
+ {
+ ref_ptr<dependency_t> &dep = dependencies[*i];
+ if (dep->targets.empty()) dep->targets.push_back(*i);
+ dep->deps.insert(rule.deps.begin(), rule.deps.end());
+ }
+}
+
+/**
+ * Register a specific rule with a nonempty script:
+ *
+ * - Check that none of the targets already has an associated rule.
+ * - Create a single shared rule and associate it to all the targets.
+ * - Merge the prerequisites of all the targets into a single set and
+ * add the prerequisites of the rule to it. (The preexisting
+ * prerequisites, if any, come from a previous run.)
+ */
+static void register_scripted_rule(rule_t const &rule)
+{
+ ref_ptr<rule_t> r(rule);
+ for (string_list::const_iterator i = rule.targets.begin(),
+ i_end = rule.targets.end(); i != i_end; ++i)
+ {
+ std::pair<rule_map::iterator, bool> j =
+ specific_rules.insert(std::make_pair(*i, r));
+ if (j.second) continue;
+ std::cerr << "Failed to load rules: " << *i
+ << " cannot be the target of several rules" << std::endl;
+ exit(EXIT_FAILURE);
+ }
+
+ ref_ptr<dependency_t> dep;
+ dep->targets = rule.targets;
+ dep->deps.insert(rule.deps.begin(), rule.deps.end());
+ for (string_list::const_iterator i = rule.targets.begin(),
+ i_end = rule.targets.end(); i != i_end; ++i)
+ {
+ ref_ptr<dependency_t> &d = dependencies[*i];
+ dep->deps.insert(d->deps.begin(), d->deps.end());
+ d = dep;
}
- load_dependencies(in);
}
/**
rule_t rule;
// Read targets and check genericity.
- string_list targets = read_words(in);
+ string_list targets;
+ if (!read_words(in, targets)) goto error;
if (!first.empty()) targets.push_front(first);
else if (targets.empty()) goto error;
else DEBUG << "actual target: " << targets.front() << std::endl;
bool generic = false;
- normalize_list(targets);
+ normalize_list(targets, "", "");
for (string_list::const_iterator i = targets.begin(),
i_end = targets.end(); i != i_end; ++i)
{
skip_spaces(in);
if (in.get() != ':') goto error;
+ bool assignment = false;
+
// Read dependencies.
- rule.deps = read_words(in);
- normalize_list(rule.deps);
+ {
+ string_list v;
+ if (expect_token(in, Word))
+ {
+ std::string d = read_word(in);
+ if (int tok = expect_token(in, Equal | Plusequal))
+ {
+ if (!read_words(in, v)) goto error;
+ assign_t &a = rule.assigns[d];
+ a.append = tok == Plusequal;
+ a.value.swap(v);
+ assignment = true;
+ goto end_line;
+ }
+ v.push_back(d);
+ }
+
+ if (!read_words(in, v)) goto error;
+ normalize_list(v, "", "");
+ rule.deps.swap(v);
+
+ if (expect_token(in, Pipe))
+ {
+ if (!read_words(in, v)) goto error;
+ normalize_list(v, "", "");
+ rule.wdeps.swap(v);
+ }
+ }
+
+ end_line:
skip_spaces(in);
- char c = in.get();
- if (c != '\r' && c != '\n') goto error;
- skip_eol(in);
+ if (!skip_eol(in, true)) goto error;
// Read script.
std::ostringstream buf;
// Add generic rules to the correct set.
if (generic)
{
+ if (assignment) goto error;
generic_rules.push_back(rule);
return;
}
- // Rules with a nonempty script lump all their targets in the same
- // dependency set, while other rules behave as if they had been
- // replicated for each of their targets.
if (!rule.script.empty())
{
- ref_ptr<dependency_t> dep;
- dep->targets = rule.targets;
- dep->deps.insert(rule.deps.begin(), rule.deps.end());
- for (string_list::const_iterator i = rule.targets.begin(),
- i_end = rule.targets.end(); i != i_end; ++i)
- {
- ref_ptr<dependency_t> &d = dependencies[*i];
- dep->deps.insert(d->deps.begin(), d->deps.end());
- d = dep;
- }
+ if (assignment) goto error;
+ register_scripted_rule(rule);
}
else
{
- for (string_list::const_iterator i = rule.targets.begin(),
- i_end = rule.targets.end(); i != i_end; ++i)
- {
- ref_ptr<dependency_t> &dep = dependencies[*i];
- if (dep->targets.empty()) dep->targets.push_back(*i);
- dep->deps.insert(rule.deps.begin(), rule.deps.end());
- }
+ // Swap away the targets to avoid costly copies when registering.
+ string_list targets;
+ std::swap(rule.targets, targets);
+ register_transparent_rule(rule, targets);
+ std::swap(rule.targets, targets);
}
+ // If there is no default target yet, mark it as such.
if (first_target.empty())
first_target = rule.targets.front();
-
- ref_ptr<rule_t> r(rule);
- for (string_list::const_iterator i = rule.targets.begin(),
- i_end = rule.targets.end(); i != i_end; ++i)
- {
- std::pair<rule_map::iterator,bool> j =
- specific_rules.insert(std::make_pair(*i, r));
- if (j.second) continue;
- std::cerr << "Failed to load rules: " << *i
- << " cannot be the target of several rules" << std::endl;
- exit(EXIT_FAILURE);
- }
-}
-
-/**
- * Save all the dependencies in file <tt>.remake</tt>.
- */
-static void save_dependencies()
-{
- DEBUG_open << "Saving database... ";
- std::ofstream db(".remake");
- while (!dependencies.empty())
- {
- ref_ptr<dependency_t> dep = dependencies.begin()->second;
- for (string_list::const_iterator i = dep->targets.begin(),
- i_end = dep->targets.end(); i != i_end; ++i)
- {
- db << escape_string(*i) << ' ';
- dependencies.erase(*i);
- }
- db << ':';
- for (string_set::const_iterator i = dep->deps.begin(),
- i_end = dep->deps.end(); i != i_end; ++i)
- {
- db << ' ' << escape_string(*i);
- }
- db << std::endl;
- }
}
/**
- * Load rules.
+ * Load rules from @a remakefile.
* If some rules have dependencies and non-generic targets, add these
* dependencies to the targets.
*/
-static void load_rules()
+static void load_rules(std::string const &remakefile)
{
DEBUG_open << "Loading rules... ";
if (false)
std::cerr << "Failed to load rules: syntax error" << std::endl;
exit(EXIT_FAILURE);
}
- std::ifstream in("Remakefile");
+ std::ifstream in(remakefile.c_str());
if (!in.good())
{
std::cerr << "Failed to load rules: no Remakefile found" << std::endl;
exit(EXIT_FAILURE);
}
- skip_eol(in);
+ skip_empty(in);
+
+ string_list options;
// Read rules
while (in.good())
if (c == '#')
{
while (in.get() != '\n') {}
- skip_eol(in);
+ skip_empty(in);
continue;
}
if (c == ' ' || c == '\t') goto error;
- token_e tok = next_token(in);
- if (tok == Word)
+ if (expect_token(in, Word))
{
std::string name = read_word(in);
if (name.empty()) goto error;
- if (next_token(in) == Equal)
+ if (int tok = expect_token(in, Equal | Plusequal))
{
- in.ignore(1);
DEBUG << "Assignment to variable " << name << std::endl;
- variables[name] = read_words(in);
- skip_eol(in);
+ string_list value;
+ if (!read_words(in, value)) goto error;
+ string_list &dest =
+ *(name == ".OPTIONS" ? &options : &variables[name]);
+ if (tok == Equal) dest.swap(value);
+ else dest.splice(dest.end(), value);
+ if (!skip_eol(in, true)) goto error;
}
else load_rule(in, name);
}
- else if (tok == Dollar)
- load_rule(in, std::string());
- else goto error;
+ else load_rule(in, std::string());
}
- // Generate script for variable assignment
- std::ostringstream buf;
- for (variable_map::const_iterator i = variables.begin(),
- i_end = variables.end(); i != i_end; ++i)
+ // Set actual options.
+ for (string_list::const_iterator i = options.begin(),
+ i_end = options.end(); i != i_end; ++i)
{
- std::ostringstream var;
- bool first = true;
- for (string_list::const_iterator j = i->second.begin(),
- j_end = i->second.end(); j != j_end; ++j)
+ if (*i == "variable-propagation") propagate_vars = true;
+ else
+ {
+ std::cerr << "Failed to load rules: unrecognized option" << std::endl;
+ exit(EXIT_FAILURE);
+ }
+ }
+}
+
+/** @} */
+
+/**
+ * @defgroup rules Rule resolution
+ *
+ * @{
+ */
+
+static void merge_rule(rule_t &dest, rule_t const &src)
+{
+ dest.deps.insert(dest.deps.end(), src.deps.begin(), src.deps.end());
+ dest.wdeps.insert(dest.wdeps.end(), src.wdeps.begin(), src.wdeps.end());
+ for (assign_map::const_iterator i = src.assigns.begin(),
+ i_end = src.assigns.end(); i != i_end; ++i)
+ {
+ if (!i->second.append)
{
- if (first) first = false;
- else var << ' ';
- var << *j;
+ new_assign:
+ dest.assigns[i->first] = i->second;
+ continue;
}
- buf << i->first << '=' << escape_string(var.str()) << std::endl;
+ assign_map::iterator j = dest.assigns.find(i->first);
+ if (j == dest.assigns.end()) goto new_assign;
+ j->second.value.insert(j->second.value.end(),
+ i->second.value.begin(), i->second.value.end());
}
- variable_block = buf.str();
}
/**
i_end = src.end(); i != i_end; ++i)
{
size_t pos = i->find('%');
- if (pos == std::string::npos)dst.push_back(*i);
+ if (pos == std::string::npos) dst.push_back(*i);
else dst.push_back(i->substr(0, pos) + pat + i->substr(pos + 1));
}
}
* - the one leading to shorter matches has priority,
* - among equivalent rules, the earliest one has priority.
*/
-static rule_t find_generic_rule(std::string const &target)
+static void find_generic_rule(job_t &job, std::string const &target)
{
size_t tlen = target.length(), plen = tlen + 1;
- rule_t rule;
for (rule_list::const_iterator i = generic_rules.begin(),
i_end = generic_rules.end(); i != i_end; ++i)
{
j->compare(pos + 1, len2, target, tlen - len2, len2))
continue;
plen = tlen - (len - 1);
- std::string pat = target.substr(pos, plen);
- rule = rule_t();
- rule.script = i->script;
- substitute_pattern(pat, i->targets, rule.targets);
- substitute_pattern(pat, i->deps, rule.deps);
+ job.stem = target.substr(pos, plen);
+ job.rule = rule_t();
+ job.rule.script = i->script;
+ substitute_pattern(job.stem, i->targets, job.rule.targets);
+ substitute_pattern(job.stem, i->deps, job.rule.deps);
+ substitute_pattern(job.stem, i->wdeps, job.rule.wdeps);
break;
}
}
- return rule;
}
/**
* If there is both a specific rule with an empty script and a generic rule, the
* generic one is returned after adding the dependencies of the specific one.
*/
-static rule_t find_rule(std::string const &target)
+static void find_rule(job_t &job, std::string const &target)
{
rule_map::const_iterator i = specific_rules.find(target),
i_end = specific_rules.end();
// If there is a specific rule with a script, return it.
- if (i != i_end && !i->second->script.empty()) return *i->second;
- rule_t grule = find_generic_rule(target);
- // If there is no generic rule, return the specific rule (no script), if any.
- if (grule.targets.empty())
+ if (i != i_end && !i->second->script.empty())
{
- if (i != i_end) return *i->second;
- return grule;
+ job.rule = *i->second;
+ return;
}
- // Optimize the lookup when there is only one target (alread looked up).
- if (grule.targets.size() == 1)
+ find_generic_rule(job, target);
+ // If there is no generic rule, return the specific rule (no script), if any.
+ if (job.rule.targets.empty())
{
if (i != i_end)
- grule.deps.insert(grule.deps.end(),
- i->second->deps.begin(), i->second->deps.end());
- return grule;
+ {
+ job.rule = *i->second;
+ return;
+ }
+ }
+ // Optimize the lookup when there is only one target (already looked up).
+ if (job.rule.targets.size() == 1)
+ {
+ if (i == i_end) return;
+ merge_rule(job.rule, *i->second);
+ return;
}
// Add the dependencies of the specific rules of every target to the
// generic rule. If any of those rules has a nonempty script, error out.
- for (string_list::const_iterator j = grule.targets.begin(),
- j_end = grule.targets.end(); j != j_end; ++j)
+ for (string_list::const_iterator j = job.rule.targets.begin(),
+ j_end = job.rule.targets.end(); j != j_end; ++j)
{
i = specific_rules.find(*j);
if (i == i_end) continue;
- if (!i->second->script.empty()) return rule_t();
- grule.deps.insert(grule.deps.end(),
- i->second->deps.begin(), i->second->deps.end());
+ if (!i->second->script.empty()) return;
+ merge_rule(job.rule, *i->second);
}
- return grule;
}
+/** @} */
+
+/**
+ * @defgroup status Target status
+ *
+ * @{
+ */
+
/**
* Compute and memoize the status of @a target:
* - if the file does not exist, the target is obsolete,
* - if any dependency is obsolete or younger than the file, it is obsolete,
* - otherwise it is up-to-date.
*
- * @note With multiple targets, they all share the same status. (If one is
- * obsolete, they all are.) For the second rule above, the latest target
- * is chosen, not the oldest!
+ * @note For rules with multiple targets, all the targets share the same
+ * status. (If one is obsolete, they all are.) The second rule above
+ * is modified in that case: the latest target is chosen, not the oldest!
*/
static status_t const &get_status(std::string const &target)
{
status[*k].last = s.st_mtime;
if (s.st_mtime > latest) latest = s.st_mtime;
}
- if (st == Todo) goto update;
+ if (st != Uptodate) goto update;
for (string_set::const_iterator k = dep.deps.begin(),
k_end = dep.deps.end(); k != k_end; ++k)
{
st = Todo;
goto update;
}
- if (ts_.status == Uptodate) continue;
- if (st == Uptodate)
+ if (ts_.status != Uptodate && st != Recheck)
+ {
DEBUG << "obsolete dependency " << *k << std::endl;
- st = Recheck;
+ st = Recheck;
+ }
}
if (st == Uptodate) DEBUG_close << "all siblings up-to-date\n";
update:
{
DEBUG_open << "Rechecking status of " << target << "... ";
status_map::iterator i = status.find(target);
- assert (i != status.end());
+ assert(i != status.end());
status_t &ts = i->second;
ts.status = Remade;
if (ts.last >= now)
}
/**
- * Check if all the prerequisites of @a target ended being up-to-date.
+ * Check whether all the prerequisites of @a target ended being up-to-date.
*/
static bool still_need_rebuild(std::string const &target)
{
DEBUG_open << "Rechecking obsoleteness of " << target << "... ";
status_map::const_iterator i = status.find(target);
- assert (i != status.end());
+ assert(i != status.end());
if (i->second.status != Recheck) return true;
dependency_map::const_iterator j = dependencies.find(target);
assert(j != dependencies.end());
return false;
}
+/** @} */
+
+/**
+ * @defgroup server Server
+ *
+ * @{
+ */
+
/**
* Handle job completion.
*/
static void complete_job(int job_id, bool success)
{
- DEBUG_open << "Completing job " << job_id << "... ";
- job_targets_map::iterator i = job_targets.find(job_id);
- assert(i != job_targets.end());
- string_list const &targets = i->second;
+ DEBUG << "Completing job " << job_id << '\n';
+ job_map::iterator i = jobs.find(job_id);
+ assert(i != jobs.end());
+ string_list const &targets = i->second.rule.targets;
if (success)
{
+ if (show_targets) std::cout << "Finished";
for (string_list::const_iterator j = targets.begin(),
j_end = targets.end(); j != j_end; ++j)
{
update_status(*j);
+ if (show_targets) std::cout << ' ' << *j;
}
+ if (show_targets) std::cout << std::endl;
}
else
{
- DEBUG_close << "failed\n";
std::cerr << "Failed to build";
for (string_list::const_iterator j = targets.begin(),
j_end = targets.end(); j != j_end; ++j)
}
std::cerr << std::endl;
}
- job_targets.erase(i);
+ jobs.erase(i);
}
/**
- * Execute the script from @a rule.
+ * Return the script obtained by substituting variables.
*/
-static bool run_script(int job_id, rule_t const &rule)
+static std::string prepare_script(job_t const &job)
{
- if (show_targets)
+ std::string const &s = job.rule.script;
+ std::istringstream in(s);
+ std::ostringstream out;
+ size_t len = s.size();
+
+ while (!in.eof())
{
- std::cout << "Building";
- for (string_list::const_iterator i = rule.targets.begin(),
- i_end = rule.targets.end(); i != i_end; ++i)
+ size_t pos = in.tellg(), p = s.find('$', pos);
+ if (p == std::string::npos || p == len - 1) p = len;
+ out.write(&s[pos], p - pos);
+ if (p == len) break;
+ ++p;
+ switch (s[p])
+ {
+ case '$':
+ out << '$';
+ in.seekg(p + 1);
+ break;
+ case '<':
+ if (!job.rule.deps.empty())
+ out << job.rule.deps.front();
+ in.seekg(p + 1);
+ break;
+ case '^':
+ {
+ bool first = true;
+ for (string_list::const_iterator i = job.rule.deps.begin(),
+ i_end = job.rule.deps.end(); i != i_end; ++i)
+ {
+ if (first) first = false;
+ else out << ' ';
+ out << *i;
+ }
+ in.seekg(p + 1);
+ break;
+ }
+ case '@':
+ assert(!job.rule.targets.empty());
+ out << job.rule.targets.front();
+ in.seekg(p + 1);
+ break;
+ case '*':
+ out << job.stem;
+ in.seekg(p + 1);
+ break;
+ case '(':
{
- std::cout << ' ' << *i;
+ in.seekg(p - 1);
+ bool first = true;
+ input_generator gen(in, &job.vars, true);
+ while (true)
+ {
+ std::string w;
+ input_status s = gen.next(w);
+ if (s == SyntaxError)
+ {
+ // TODO
+ return "false";
+ }
+ if (s == Eof) break;
+ if (first) first = false;
+ else out << ' ';
+ out << w;
+ }
+ break;
+ }
+ default:
+ // Let dollars followed by an unrecognized character
+ // go through. This differs from Make, which would
+ // use a one-letter variable.
+ out << '$';
+ in.seekg(p);
}
- std::cout << std::endl;
}
+ return out.str();
+}
+
+/**
+ * Execute the script from @a rule.
+ */
+static status_e run_script(int job_id, job_t const &job)
+{
ref_ptr<dependency_t> dep;
- dep->targets = rule.targets;
- dep->deps.insert(rule.deps.begin(), rule.deps.end());
- for (string_list::const_iterator i = rule.targets.begin(),
- i_end = rule.targets.end(); i != i_end; ++i)
+ dep->targets = job.rule.targets;
+ dep->deps.insert(job.rule.deps.begin(), job.rule.deps.end());
+ if (show_targets) std::cout << "Building";
+ for (string_list::const_iterator i = job.rule.targets.begin(),
+ i_end = job.rule.targets.end(); i != i_end; ++i)
{
dependencies[*i] = dep;
+ if (show_targets) std::cout << ' ' << *i;
+ }
+ if (show_targets) std::cout << std::endl;
+
+ std::string script = prepare_script(job);
+
+ std::ostringstream job_id_buf;
+ job_id_buf << job_id;
+ std::string job_id_ = job_id_buf.str();
+
+ DEBUG_open << "Starting script for job " << job_id << "... ";
+ if (script.empty())
+ {
+ DEBUG_close << "no script\n";
+ complete_job(job_id, true);
+ return Remade;
+ }
+
+ if (false)
+ {
+ error:
+ DEBUG_close << "failed\n";
+ complete_job(job_id, false);
+ return Failed;
}
- DEBUG_open << "Starting script for job " << job_id << " with variables ("
- << (job_id >= 0 ? serialize_variables(job_variables[job_id], ' ') : "")
- << ") ... ";
#ifdef WINDOWS
HANDLE pfd[2];
if (false)
error2:
CloseHandle(pfd[0]);
CloseHandle(pfd[1]);
- error:
- DEBUG_close << "failed\n";
- complete_job(job_id, false);
- return false;
+ goto error;
}
if (!CreatePipe(&pfd[0], &pfd[1], NULL, 0))
goto error;
si.dwFlags |= STARTF_USESTDHANDLES;
PROCESS_INFORMATION pi;
ZeroMemory(&pi, sizeof(PROCESS_INFORMATION));
- std::ostringstream buf;
- buf << job_id;
- if (!SetEnvironmentVariable("REMAKE_JOB_ID", buf.str().c_str()))
+ if (!SetEnvironmentVariable("REMAKE_JOB_ID", job_id_.c_str()))
goto error2;
- std::ostringstream argv;
- argv << "SH.EXE -e -s";
- if (echo_scripts) argv << " -v";
- for (string_list::const_iterator i = rule.targets.begin(),
- i_end = rule.targets.end(); i != i_end; ++i)
- {
- argv << " \"" << escape_string(*i) << '"';
- }
- if (!CreateProcess(NULL, (char *)argv.str().c_str(), NULL, NULL,
+ char const *argv = echo_scripts ? "SH.EXE -e -s -v" : "SH.EXE -e -s";
+ if (!CreateProcess(NULL, (char *)argv, NULL, NULL,
true, 0, NULL, NULL, &si, &pi))
{
goto error2;
}
CloseHandle(pi.hThread);
- std::string script = variable_block
- + (job_id >= 0 ? serialize_variables(job_variables[job_id], '\n') : "")
- + rule.script;
DWORD len = script.length(), wlen;
if (!WriteFile(pfd[1], script.c_str(), len, &wlen, NULL) || wlen < len)
std::cerr << "Unexpected failure while sending script to shell" << std::endl;
CloseHandle(pfd[1]);
++running_jobs;
job_pids[pi.hProcess] = job_id;
- return true;
+ return Running;
#else
int pfd[2];
if (false)
error2:
close(pfd[0]);
close(pfd[1]);
- error:
- DEBUG_close << "failed\n";
- complete_job(job_id, false);
- return false;
+ goto error;
}
if (pipe(pfd) == -1)
goto error;
- if (pid_t pid = fork())
+ if (setenv("REMAKE_JOB_ID", job_id_.c_str(), 1))
+ goto error2;
+ if (pid_t pid = vfork())
{
if (pid == -1) goto error2;
- std::string script = variable_block
- + (job_id >= 0 ? serialize_variables(job_variables[job_id], '\n') : "")
- + rule.script;
ssize_t len = script.length();
if (write(pfd[1], script.c_str(), len) < len)
std::cerr << "Unexpected failure while sending script to shell" << std::endl;
close(pfd[1]);
++running_jobs;
job_pids[pid] = job_id;
- return true;
+ return Running;
}
- // Child process starts here.
- std::ostringstream buf;
- buf << job_id;
- if (setenv("REMAKE_JOB_ID", buf.str().c_str(), 1))
- _exit(EXIT_FAILURE);
- int num = echo_scripts ? 4 : 3;
- char const **argv = new char const *[num + rule.targets.size() + 1];
- argv[0] = "sh";
- argv[1] = "-e";
- argv[2] = "-s";
+ // Child process starts here. Notice the use of vfork above.
+ char const *argv[5] = { "sh", "-e", "-s", NULL, NULL };
if (echo_scripts) argv[3] = "-v";
- for (string_list::const_iterator i = rule.targets.begin(),
- i_end = rule.targets.end(); i != i_end; ++i, ++num)
- {
- argv[num] = i->c_str();
- }
- argv[num] = NULL;
+ close(pfd[1]);
if (pfd[0] != 0)
{
dup2(pfd[0], 0);
close(pfd[0]);
}
- close(pfd[1]);
- execv("/bin/sh", (char **)argv);
+ execve("/bin/sh", (char **)argv, environ);
_exit(EXIT_FAILURE);
#endif
}
/**
* Create a job for @a target according to the loaded rules.
* Mark all the targets from the rule as running and reset their dependencies.
+ * Inherit variables from @a current, if enabled.
* If the rule has dependencies, create a new client to build them just
* before @a current, and change @a current so that it points to it.
*/
-static bool start(std::string const &target, client_list::iterator ¤t)
+static status_e start(std::string const &target, client_list::iterator ¤t)
{
- DEBUG_open << "Starting job " << job_counter << " for " << target << "... ";
- rule_t rule = find_rule(target);
- if (rule.targets.empty())
+ int job_id = job_counter++;
+ DEBUG_open << "Starting job " << job_id << " for " << target << "... ";
+ job_t &job = jobs[job_id];
+ find_rule(job, target);
+ if (job.rule.targets.empty())
{
status[target].status = Failed;
DEBUG_close << "failed\n";
std::cerr << "No rule for building " << target << std::endl;
- return false;
+ return Failed;
}
- for (string_list::const_iterator i = rule.targets.begin(),
- i_end = rule.targets.end(); i != i_end; ++i)
+ for (string_list::const_iterator i = job.rule.targets.begin(),
+ i_end = job.rule.targets.end(); i != i_end; ++i)
{
status[*i].status = Running;
}
- int job_id = job_counter++;
- job_targets[job_id] = rule.targets;
- DEBUG << "Setting variables of job: " << job_id << " (spawn by: "
- << current->job_id << ") to "
- << serialize_variables(current->variables, ' ') << std::endl;
- job_variables[job_id] = current->variables;
- if (!rule.deps.empty())
- {
- DEBUG << "Current client has job_id: " << job_id
- << " and variables " << serialize_variables(current->variables, ' ')
- << std::endl;
- client_t dep_client = client_t();
- dep_client.variables = current->variables;
- current = clients.insert(current, dep_client);
- current->job_id = job_id;
- current->pending = rule.deps;
- current->delayed = new rule_t(rule);
- return true;
+ if (propagate_vars) job.vars = current->vars;
+ for (assign_map::const_iterator i = job.rule.assigns.begin(),
+ i_end = job.rule.assigns.end(); i != i_end; ++i)
+ {
+ std::pair<variable_map::iterator, bool> k =
+ job.vars.insert(std::make_pair(i->first, string_list()));
+ string_list &v = k.first->second;
+ if (i->second.append)
+ {
+ if (k.second)
+ {
+ variable_map::const_iterator j = variables.find(i->first);
+ if (j != variables.end()) v = j->second;
+ }
+ }
+ else if (!k.second) v.clear();
+ v.insert(v.end(), i->second.value.begin(), i->second.value.end());
}
- return run_script(job_id, rule);
+ if (!job.rule.deps.empty() || !job.rule.wdeps.empty())
+ {
+ current = clients.insert(current, client_t());
+ current->job_id = job_id;
+ current->pending = job.rule.deps;
+ current->pending.insert(current->pending.end(),
+ job.rule.wdeps.begin(), job.rule.wdeps.end());
+ if (propagate_vars) current->vars = job.vars;
+ current->delayed = true;
+ return Recheck;
+ }
+ return run_script(job_id, job);
}
/**
assert(client.socket == INVALID_SOCKET);
if (success)
{
- if (still_need_rebuild(client.delayed->targets.front()))
- run_script(client.job_id, *client.delayed);
+ job_map::const_iterator i = jobs.find(client.job_id);
+ assert(i != jobs.end());
+ if (still_need_rebuild(i->second.rule.targets.front()))
+ run_script(client.job_id, i->second);
else complete_job(client.job_id, true);
}
else complete_job(client.job_id, false);
- delete client.delayed;
}
else if (client.socket != INVALID_SOCKET)
{
char res = success ? 1 : 0;
- send(client.socket, &res, 1, 0);
+ send(client.socket, &res, 1, MSG_NOSIGNAL);
#ifdef WINDOWS
closesocket(client.socket);
#else
}
/**
- * Update clients as long as there are free slots:
+ * Handle client requests:
* - check for running targets that have finished,
* - start as many pending targets as allowed,
* - complete the request if there are neither running nor pending targets
* left or if any of them failed.
+ *
+ * @return true if some child processes are still running.
+ *
+ * @post If there are pending requests, at least one child process is running.
+ *
+ * @invariant New free slots cannot appear during a run, since the only way to
+ * decrease #running_jobs is #finalize_job and the only way to
+ * increase #waiting_jobs is #accept_client. None of these functions
+ * are called during a run. So breaking out as soon as there no free
+ * slots left is fine.
*/
-static void update_clients()
+static bool handle_clients()
{
- DEBUG_open << "Updating clients... ";
+ DEBUG_open << "Handling client requests... ";
+ restart:
+ bool need_restart = false;
+
for (client_list::iterator i = clients.begin(), i_next = i,
i_end = clients.end(); i != i_end && has_free_slots(); i = i_next)
{
++i_next;
DEBUG_open << "Handling client from job " << i->job_id << "... ";
- if (false)
- {
- failed:
- complete_request(*i, false);
- clients.erase(i);
- DEBUG_close << "failed\n";
- continue;
- }
// Remove running targets that have finished.
for (string_set::iterator j = i->running.begin(), j_next = j,
case Running:
break;
case Failed:
- if (!keep_going) goto failed;
i->failed = true;
+ if (!keep_going) goto complete;
// no break
case Uptodate:
case Remade:
break;
case Failed:
pending_failed:
- if (!keep_going) goto failed;
i->failed = true;
+ if (!keep_going) goto complete;
// no break
case Uptodate:
case Remade:
case Recheck:
case Todo:
client_list::iterator j = i;
- if (!start(target, i)) goto pending_failed;
- j->running.insert(target);
- if (!has_free_slots()) return;
- // Job start might insert a dependency client.
- i_next = i;
- ++i_next;
- break;
+ switch (start(target, i))
+ {
+ case Failed:
+ goto pending_failed;
+ case Running:
+ // A shell was started, check for free slots.
+ j->running.insert(target);
+ if (!has_free_slots()) return true;
+ break;
+ case Recheck:
+ // Switch to the dependency client that was inserted.
+ j->running.insert(target);
+ i_next = j;
+ break;
+ case Remade:
+ // Nothing to run.
+ need_restart = true;
+ break;
+ default:
+ assert(false);
+ }
}
}
- // Try to complete request.
+ // Try to complete the request.
// (This might start a new job if it was a dependency client.)
- if (i->running.empty())
+ if (i->running.empty() || i->failed)
{
- if (i->failed) goto failed;
- complete_request(*i, true);
+ complete:
+ complete_request(*i, !i->failed);
+ DEBUG_close << (i->failed ? "failed\n" : "finished\n");
clients.erase(i);
- DEBUG_close << "finished\n";
+ need_restart = true;
}
}
+
+ if (running_jobs != waiting_jobs) return true;
+ if (running_jobs == 0 && clients.empty()) return false;
+ if (need_restart) goto restart;
+
+ // There is a circular dependency.
+ // Try to break it by completing one of the requests.
+ assert(!clients.empty());
+ std::cerr << "Circular dependency detected" << std::endl;
+ client_list::iterator i = clients.begin();
+ complete_request(*i, false);
+ clients.erase(i);
+ goto restart;
}
/**
// Create and listen to the socket.
socket_fd = socket(AF_INET, SOCK_STREAM, 0);
- if (socket_fd < 0) goto error;
+ if (socket_fd == INVALID_SOCKET) goto error;
if (!SetHandleInformation((HANDLE)socket_fd, HANDLE_FLAG_INHERIT, 0))
goto error;
if (bind(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
goto error;
if (listen(socket_fd, 1000)) goto error;
#else
- // Set a handler for SIGCHLD then block the signal (unblocked during select).
+ // Set signal handlers for SIGCHLD and SIGINT.
+ // Block SIGCHLD (unblocked during select).
sigset_t sigmask;
sigemptyset(&sigmask);
sigaddset(&sigmask, SIGCHLD);
if (sigprocmask(SIG_BLOCK, &sigmask, NULL) == -1) goto error;
struct sigaction sa;
sa.sa_flags = 0;
- sa.sa_handler = &child_sig_handler;
sigemptyset(&sa.sa_mask);
+ sa.sa_handler = &sigchld_handler;
if (sigaction(SIGCHLD, &sa, NULL) == -1) goto error;
+ sa.sa_handler = &sigint_handler;
+ if (sigaction(SIGINT, &sa, NULL) == -1) goto error;
// Prepare a named unix socket in temporary directory.
socket_name = tempnam(NULL, "rmk-");
// Create and listen to the socket.
#ifdef LINUX
socket_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
- if (socket_fd < 0) goto error;
+ if (socket_fd == INVALID_SOCKET) goto error;
#else
socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
- if (socket_fd < 0) goto error;
+ if (socket_fd == INVALID_SOCKET) goto error;
if (fcntl(socket_fd, F_SETFD, FD_CLOEXEC) < 0) goto error;
#endif
if (bind(socket_fd, (struct sockaddr *)&socket_addr, len))
* Accept a connection from a client, get the job it spawned from,
* get the targets, and mark them as dependencies of the job targets.
*/
-void accept_client()
+static void accept_client()
{
DEBUG_open << "Handling client request... ";
memcpy(&job_id, &buf[0], sizeof(int));
proc->socket = fd;
proc->job_id = job_id;
- job_targets_map::const_iterator i = job_targets.find(job_id);
- if (i == job_targets.end()) goto error;
+ job_map::const_iterator i = jobs.find(job_id);
+ if (i == jobs.end()) goto error;
DEBUG << "receiving request from job " << job_id << std::endl;
+ if (propagate_vars) proc->vars = i->second.vars;
- // Parse the targets and mark them as dependencies from the job targets.
- dependency_t &dep = *dependencies[job_targets[job_id].front()];
+ // Parse the targets and the variable assignments.
+ // Mark the targets as dependencies of the job targets.
+ dependency_t &dep = *dependencies[i->second.rule.targets.front()];
+ string_list *last_var = NULL;
char const *p = &buf[0] + sizeof(int);
while (true)
{
len = strlen(p);
- if (len == 1 && p[0] == 1)
+ if (len == 0)
{
- //Finished parsing targets.
- p += 2;
++waiting_jobs;
- break;
+ break;
+ }
+ switch (*p)
+ {
+ case 'T':
+ {
+ if (len == 1) goto error;
+ std::string target(p + 1, p + len);
+ DEBUG << "adding dependency " << target << " to job\n";
+ proc->pending.push_back(target);
+ dep.deps.insert(target);
+ break;
+ }
+ case 'V':
+ {
+ if (len == 1) goto error;
+ std::string var(p + 1, p + len);
+ DEBUG << "adding variable " << var << " to job\n";
+ last_var = &proc->vars[var];
+ last_var->clear();
+ break;
+ }
+ case 'W':
+ {
+ if (!last_var) goto error;
+ last_var->push_back(std::string(p + 1, p + len));
+ break;
+ }
+ default:
+ goto error;
}
- std::string target(p, p + len);
- DEBUG << "adding dependency " << target << " to job " << job_id << std::endl;
- proc->pending.push_back(target);
- dep.deps.insert(target);
p += len + 1;
}
- while (true)
- {
- len = strlen(p);
- if (len == 0) return;
- std::string line(p, p + len);
- std::istringstream in (line);
- std::string name = read_word(in);
- if (next_token(in) != Equal) {
- std::cerr << '\'' << line << "'" << std::endl;
- goto error;
- }
- in.ignore(1); // ignore =.
- DEBUG << "adding variable " << line << " to job " << job_id << std::endl;
- string_list l = read_words(in);
- proc->variables[name] = l;
- p += len + 1;
- }
-
+ if (!propagate_vars && !proc->vars.empty())
+ {
+ std::cerr << "Assignments are ignored unless 'variable-propagation' is enabled" << std::endl;
+ proc->vars.clear();
+ }
+}
+/**
+ * Handle child process exit status.
+ */
+static void finalize_job(pid_t pid, bool res)
+{
+ pid_job_map::iterator i = job_pids.find(pid);
+ assert(i != job_pids.end());
+ int job_id = i->second;
+ job_pids.erase(i);
+ --running_jobs;
+ complete_job(job_id, res);
}
/**
* Loop until all the jobs have finished.
+ *
+ * @post There are no client requests left, not even virtual ones.
*/
-void server_loop()
+static void server_loop()
{
- while (true)
+ while (handle_clients())
{
- update_clients();
- if (running_jobs == 0)
- {
- assert(clients.empty());
- break;
- }
DEBUG_open << "Handling events... ";
#ifdef WINDOWS
size_t len = job_pids.size() + 1;
DWORD w = WaitForMultipleObjects(len, h, false, INFINITE);
WSAEventSelect(socket_fd, aev, 0);
WSACloseEvent(aev);
- if (w < WAIT_OBJECT_0 || WAIT_OBJECT_0 + len <= w)
+ if (len <= w)
continue;
- if (w == WAIT_OBJECT_0 + len - 1)
+ if (w == len - 1)
{
accept_client();
continue;
}
- pid_t pid = h[w - WAIT_OBJECT_0];
+ pid_t pid = h[w];
DWORD s = 0;
bool res = GetExitCodeProcess(pid, &s) && s == 0;
CloseHandle(pid);
- pid_job_map::iterator i = job_pids.find(pid);
- assert(i != job_pids.end());
- int job_id = i->second;
- job_pids.erase(i);
- --running_jobs;
- complete_job(job_id, res);
+ finalize_job(pid, res);
#else
sigset_t emptymask;
sigemptyset(&emptymask);
while ((pid = waitpid(-1, &status, WNOHANG)) > 0)
{
bool res = WIFEXITED(status) && WEXITSTATUS(status) == 0;
- pid_job_map::iterator i = job_pids.find(pid);
- assert(i != job_pids.end());
- int job_id = i->second;
- job_pids.erase(i);
- --running_jobs;
- complete_job(job_id, res);
+ finalize_job(pid, res);
}
#endif
}
+
+ assert(clients.empty());
}
/**
* If Remakefile is obsolete, perform a first run with it only, then reload
* the rules, and perform a second with the original clients.
*/
-void server_mode(string_list const &targets)
+static void server_mode(std::string const &remakefile, string_list const &targets)
{
load_dependencies();
- load_rules();
+ load_rules(remakefile);
create_server();
- if (get_status("Remakefile").status != Uptodate)
+ if (get_status(remakefile).status != Uptodate)
{
clients.push_back(client_t());
- clients.back().pending.push_back("Remakefile");
+ clients.back().pending.push_back(remakefile);
server_loop();
if (build_failure) goto early_exit;
variables.clear();
specific_rules.clear();
generic_rules.clear();
first_target.clear();
- load_rules();
+ load_rules(remakefile);
}
clients.push_back(client_t());
if (!targets.empty()) clients.back().pending = targets;
server_loop();
early_exit:
close(socket_fd);
+#ifndef WINDOWS
remove(socket_name);
+ free(socket_name);
+#endif
save_dependencies();
+ if (show_targets && changed_prefix_dir)
+ {
+ std::cout << "remake: Leaving directory `" << prefix_dir << '\'' << std::endl;
+ }
exit(build_failure ? EXIT_FAILURE : EXIT_SUCCESS);
}
+/** @} */
+
+/**
+ * @defgroup client Client
+ *
+ * @{
+ */
+
/**
- * Connect to the server @a socket_name, send a build request for @a targets,
- * and exit with the status returned by the server.
+ * Connect to the server @a socket_name, send a request for building @a targets
+ * with some @a variables, and exit with the status returned by the server.
*/
-void client_mode(char *socket_name, string_list const &targets)
+static void client_mode(char *socket_name, string_list const &targets)
{
if (false)
{
#ifdef WINDOWS
struct sockaddr_in socket_addr;
socket_fd = socket(AF_INET, SOCK_STREAM, 0);
- if (socket_fd < 0) goto error;
+ if (socket_fd == INVALID_SOCKET) goto error;
socket_addr.sin_family = AF_INET;
socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
socket_addr.sin_port = atoi(socket_name);
size_t len = strlen(socket_name);
if (len >= sizeof(socket_addr.sun_path) - 1) exit(EXIT_FAILURE);
socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
- if (socket_fd < 0) goto error;
+ if (socket_fd == INVALID_SOCKET) goto error;
socket_addr.sun_family = AF_UNIX;
strcpy(socket_addr.sun_path, socket_name);
if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(socket_addr.sun_family) + len))
for (string_list::const_iterator i = targets.begin(),
i_end = targets.end(); i != i_end; ++i)
{
- DEBUG_open << "Sending " << *i << "... ";
- ssize_t len = i->length() + 1;
- if (send(socket_fd, i->c_str(), len, MSG_NOSIGNAL) != len)
+ DEBUG_open << "Sending target " << *i << "... ";
+ std::string s = 'T' + *i;
+ ssize_t len = s.length() + 1;
+ if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
goto error;
}
- // send 10 as as separator between targets and variables.
- char result = 1;
- if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
- result = 0;
- if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
- // Send variables.
- // (maybe split vars in small chunks and send... seems like an overkil)
- std::string vars = serialize_variables(variables, 0);
- ssize_t sent = vars.size();
- DEBUG << "Sending variables: '" << vars << "' to the server" << std::endl;
- if (send(socket_fd, vars.data(), sent, MSG_NOSIGNAL) != sent)
- goto error;
+
+ // Send variables.
+ for (variable_map::const_iterator i = variables.begin(),
+ i_end = variables.end(); i != i_end; ++i)
+ {
+ DEBUG_open << "Sending variable " << i->first << "... ";
+ std::string s = 'V' + i->first;
+ ssize_t len = s.length() + 1;
+ if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
+ goto error;
+ for (string_list::const_iterator j = i->second.begin(),
+ j_end = i->second.end(); j != j_end; ++j)
+ {
+ std::string s = 'W' + *j;
+ len = s.length() + 1;
+ if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
+ goto error;
+ }
+ }
// Send terminating nul and wait for reply.
+ char result = 0;
if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
if (recv(socket_fd, &result, 1, 0) != 1) exit(EXIT_FAILURE);
exit(result ? EXIT_SUCCESS : EXIT_FAILURE);
}
+/** @} */
+
+/**
+ * @defgroup ui User interface
+ *
+ * @{
+ */
+
/**
* Display usage and exit with @a exit_status.
*/
-void usage(int exit_status)
+static void usage(int exit_status)
{
std::cerr << "Usage: remake [options] [target] ...\n"
"Options\n"
" -d Echo script commands.\n"
" -d -d Print lots of debugging information.\n"
+ " -f FILE Read FILE as Remakefile.\n"
" -h, --help Print this message and exit.\n"
" -j[N], --jobs=[N] Allow N jobs at once; infinite jobs with no arg.\n"
" -k Keep going when some targets cannot be made.\n"
- " -r Look up targets from the dependencies on standard input.\n"
- " -v V=X, --var V=X Initialize variable V with X"
+ " -r Look up targets from the dependencies on stdin.\n"
" -s, --silent, --quiet Do not echo targets.\n";
exit(exit_status);
}
*/
int main(int argc, char *argv[])
{
- init_working_dir(argv[0]);
-
+ std::string remakefile;
string_list targets;
+ bool literal_targets = false;
bool indirect_targets = false;
// Parse command-line arguments.
{
std::string arg = argv[i];
if (arg.empty()) usage(EXIT_FAILURE);
+ if (literal_targets) goto new_target;
if (arg == "-h" || arg == "--help") usage(EXIT_SUCCESS);
if (arg == "-d")
if (echo_scripts) debug.active = true;
show_targets = false;
else if (arg == "-r")
indirect_targets = true;
+ else if (arg == "-f")
+ {
+ if (++i == argc) usage(EXIT_FAILURE);
+ remakefile = argv[i];
+ }
+ else if (arg == "--")
+ literal_targets = true;
else if (arg.compare(0, 2, "-j") == 0)
max_active_jobs = atoi(arg.c_str() + 2);
else if (arg.compare(0, 7, "--jobs=") == 0)
max_active_jobs = atoi(arg.c_str() + 7);
- else if (arg == "-v" || arg == "--var")
- {
- ++i;
- if (i == argc) usage(EXIT_FAILURE);
- arg = argv[i];
- std::istringstream in (arg);
- std::string name = read_word(in);
- if (next_token(in) != Equal) {
- std::cerr << "Invalid variable '" << arg << "'" << std::endl;
- exit(EXIT_FAILURE);
- }
- in.ignore(1);
- string_list l = read_words(in);
- variables[name] = l;
- }
else
{
- if (arg[0] == '-') usage(1);
- targets.push_back(normalize(arg));
+ if (arg[0] == '-') usage(EXIT_FAILURE);
+ if (arg.find('=') != std::string::npos)
+ {
+ std::istringstream in(arg);
+ std::string name = read_word(in);
+ if (name.empty() || !expect_token(in, Equal)) usage(EXIT_FAILURE);
+ read_words(in, variables[name]);
+ continue;
+ }
+ new_target:
+ targets.push_back(arg);
DEBUG << "New target: " << arg << '\n';
}
}
+ init_working_dir();
+ normalize_list(targets, working_dir, working_dir);
+
if (indirect_targets)
{
load_dependencies(std::cin);
for (string_set::const_iterator k = dep.deps.begin(),
k_end = dep.deps.end(); k != k_end; ++k)
{
- targets.push_back(normalize(*k));
+ targets.push_back(normalize(*k, working_dir, working_dir));
}
}
dependencies.clear();
if (char *sn = getenv("REMAKE_SOCKET")) client_mode(sn, targets);
// Otherwise run as server.
- server_mode(targets);
+ if (remakefile.empty())
+ {
+ remakefile = "Remakefile";
+ init_prefix_dir();
+ }
+ normalize_list(targets, working_dir, prefix_dir);
+ server_mode(remakefile, targets);
}
+
+/** @} */
projects / tatoo.git / blobdiff