2 @mainpage Remake, a build system that bridges the gap between make and redo.
4 As with <b>make</b>, <b>remake</b> uses a centralized rule file, which is
5 named <b>Remakefile</b>. It contains rules with a <em>make</em>-like
9 target1 target2 ... : dependency1 dependency2 ...
15 A target is known to be up-to-date if all its dependencies are. If it
16 has no known dependencies yet the file already exits, it is assumed to
17 be up-to-date. Obsolete targets are rebuilt thanks to the shell script
20 As with <b>redo</b>, <b>remake</b> supports dynamic dependencies in
21 addition to these static dependencies. Whenever a script executes
22 <tt>remake dependency4 dependency5 ...</tt>, these dependencies are
23 rebuilt if they are obsolete. (So <b>remake</b> acts like
24 <b>redo-ifchange</b>.) Moreover, these dependencies are stored in file
25 <b>.remake</b> so that they are remembered in subsequent runs. Note that
26 dynamic dependencies from previous runs are only used to decide whether a
27 target is obsolete; they are not automatically rebuilt when they are
28 obsolete yet a target depends on them. They will only be rebuilt once the
29 dynamic call to <b>remake</b> is executed.
31 In other words, the following two rules have almost the same behavior.
34 target1 target2 ... : dependency1 dependency2 ...
38 remake dependency1 dependency2 ...
42 (There is a difference if the targets already exist, have never been
43 built before, and the dependencies are either younger or obsolete, since
44 the targets will not be rebuilt in the second case.)
46 The above usage of dynamic dependencies is hardly useful. Their strength
47 lies in the fact that they can be computed on the fly:
51 gcc -MMD -MF $1.d -o $1 -c ${1%.o}.c
56 ocamldep ${1%.cmo}.ml | remake -r $1
57 ocamlc -c ${1%.cmo}.ml
59 after.xml: before.xml rules.xsl
60 xsltproc --load-trace -o after.xml rules.xsl before.xml 2> deps
61 remake $(sed -n -e "\\,//,! s,^.*URL=\"\\([^\"]*\\).*\$,\\1,p" deps)
65 Note that the first rule fails if any of the header files included by
66 a C source file has to be automatically generated. In that case, one
67 should perform a first call to <b>remake</b> them before calling the
68 compiler. (Dependencies from several calls to <b>remake</b> are
69 cumulative, so they will all be remembered the next time.)
71 \section sec-usage Usage
73 Usage: <tt>remake <i>options</i> <i>targets</i></tt>
77 - <tt>-d</tt>: Echo script commands.
78 - <tt>-j[N]</tt>, <tt>--jobs=[N]</tt>: Allow N jobs at once; infinite jobs
80 - <tt>-k</tt>, <tt>--keep-going</tt>: Keep going when some targets cannot be made.
81 - <tt>-r</tt>: Look up targets from the dependencies on standard input.
82 - <tt>-s</tt>, <tt>--silent</tt>, <tt>--quiet</tt>: Do not echo targets.
84 \section sec-syntax Syntax
86 Lines starting with a space character or a tabulation are assumed to be rule
87 scripts. They are only allowed after a rule header.
89 Lines starting with <tt>#</tt> are considered to be comments and are ignored.
90 They do interrupt rule scripts though.
92 Any other line is either a rule header or a variable definition. If such a
93 line ends with a backslash, the following line break is ignored and the line
94 extends to the next one.
96 Rule headers are a nonempty list of names, followed by a colon, followed by
97 another list of names, possibly empty. Variable definitions are a single
98 name followed by equal followed by a list of names, possibly empty. Basically,
99 the syntax of a rule is as follows:
102 targets : prerequisites
106 List of names are space-separated sequences of names. If a name contains a
107 space character, it should be put into double quotes. Names can not be any
108 of the following special characters <tt>:$(),="</tt>. Again, quotation
109 should be used. Quotation marks can be escaped by a backslash inside
112 \subsection sec-variables Variables
114 Variables can be used to factor lists of targets or dependencies. They are
115 expanded as they are encountered during <b>Remakefile</b> parsing.
123 Variables can be used inside rule scripts; they are available as non-exported
124 shell variables there.
126 \subsection sec-functions Built-in functions
128 <b>remake</b> also supports a few built-in functions inspired from <b>make</b>.
130 - <tt>$(addprefix <i>prefix</i>, <i>list</i>)</tt> returns the list obtained
131 by prepending its first argument to each element of its second argument.
132 - <tt>$(addsuffix <i>suffix</i>, <i>list</i>)</tt> returns the list obtained
133 by appending its first argument to each element of its second argument.
135 Note that functions are ignored inside scripts.
137 \section sec-semantics Semantics
139 \subsection src-obsolete When are targets obsolete?
141 A target is obsolete:
143 - if there is no file corresponding to the target, or to one of its siblings
144 in a multi-target rule,
145 - if any of its dynamic dependencies from a previous run or any of its static
146 prerequisites is obsolete,
147 - if the latest file corresponding to its siblings or itself is older than any
148 of its dynamic dependencies or static prerequisites.
150 In all the other cases, it is assumed to be up-to-date (and so are all its
151 siblings). Note that the last rule above says "latest" and not "earliest". While
152 it might cause some obsolete targets to go unnoticed in corner cases, it allows
153 for the following kind of rules:
156 config.h stamp-config_h: config.h.in config.status
157 ./config.status config.h
161 A <tt>config.status</tt> file generally does not update header files (here
162 <tt>config.h</tt>) if they would not change. As a consequence, if not for the
163 <tt>stamp-config_h</tt> file above, a header would always be considered obsolete
164 once one of its prerequisites is modified. Note that touching <tt>config.h</tt>
165 rather than <tt>stamp-config_h</tt> would defeat the point of not updating it
166 in the first place, since the program files would need to be rebuilt.
168 Once all the static prerequisites of a target have been rebuilt, <b>remake</b>
169 checks if the target still needs to be built. If it was obsolete only because
170 its dependencies needed to be rebuilt and none of them changed, the target is
171 assumed to be up-to-date.
173 \subsection sec-rules How are targets (re)built?
175 There are two kinds of rules. If any of the targets or prerequisites contains
176 a <tt>%</tt> character, the rule is said to be <em>generic</em>. All the
177 targets of the rule shall then contain a single <tt>%</tt> character. All the
178 other rules are said to be <em>specific</em>.
180 A rule is said to <em>match</em> a given target:
182 - if it is specific and the target appears inside its target list,
183 - if it is generic and there is a way to replace the <tt>%</tt> character
184 from one of its targets so that it matches the given target.
186 When <b>remake</b> tries to build a given target, it looks for a specific rule
187 that matches it. If there is one and its script is nonempty, it uses it to
190 Otherwise, it looks for a generic rule that match the target. If there are
191 several matching rules, it chooses the one with the shortest pattern (and if
192 there are several ones, the earliest one). <b>remake</b> then looks for
193 specific rules that match each target of the generic rule. All the
194 prerequisites of these specific rules are added to those of the generic rule.
195 The script of the generic rule is used to build the target.
201 commands building t%1 and t2%
204 commands building t2z
208 # t2x is built by the first rule (which also builds tx1) and its prerequisites are p1, px2
209 # t2y is built by the first rule (which also builds ty1) and its prerequisites are p1, py2, p3
210 # t2z is built by the second rule and its prerequisite is p4
213 The set of rules from <b>Remakefile</b> is ill-formed:
215 - if any specific rule matching a target of the generic rule has a nonempty script,
216 - if any target of the generic rule is matched by a generic rule with a shorter pattern.
218 \section sec-compilation Compilation
220 - On Linux, MacOSX, and BSD: <tt>g++ -o remake remake.cpp</tt>
221 - On Windows: <tt>g++ -o remake.exe remake.cpp -lws2_32</tt>
223 Installing <b>remake</b> is needed only if <b>Remakefile</b> does not
224 specify the path to the executable for its recursive calls. Thanks to its
225 single source file, <b>remake</b> can be shipped inside other packages and
226 built at configuration time.
228 \section sec-differences Differences with other build systems
230 Differences with <b>make</b>:
232 - Dynamic dependencies are supported.
233 - For rules with multiple targets, the shell script is executed only once
234 and is assumed to build all the targets. There is no need for
235 convoluted rules that are robust enough for parallel builds. For generic
236 rules, this is similar to the behavior of pattern rules from <b>gmake</b>.
237 - As with <b>redo</b>, only one shell is run when executing a script,
238 rather than one per script line. Note that the shells are run with
239 option <tt>-e</tt>, thus causing them to exit as soon as an error is
241 - The dependencies of generic rules (known as implicit rules in make lingo)
242 are not used to decide between several of them. <b>remake</b> does not
243 select one for which it could satisfy the dependencies.
244 - Variables and built-in functions are expanded as they are encountered
245 during <b>Remakefile</b> parsing.
247 Differences with <b>redo</b>:
249 - As with <b>make</b>, it is possible to write the following kind of rules
252 Remakefile: Remakefile.in ./config.status
253 ./config.status Remakefile
255 - If a target is already built the first time <b>remake</b> runs, it still
256 uses the static prerequisites of rules mentioning it to check whether it
257 needs to be rebuilt. It does not assume it to be up-to-date. As with
258 <b>redo</b> though, if its obsolete status would be due to a dynamic
259 dependency, it will go unnoticed; it should be removed beforehand.
260 - <b>remake</b> has almost no features: no checksum-based dependencies, no
261 compatibility with token servers, etc.
263 Differences with both <b>make</b> and <b>redo</b>:
265 - Multiple targets are supported.
266 - When executing shell scripts, positional variables <tt>$1</tt>,
267 <tt>$2</tt>, etc, point to the target names of the rule obtained after
268 substituting <tt>%</tt>. No other variables are defined.
270 \section sec-limitations Limitations
272 - When the user or a script calls <b>remake</b>, the current working
273 directory should be the one containing <b>Remakefile</b> (and thus
275 - Some cases of ill-formed rules are not caught by <b>remake</b> and can
276 thus lead to unpredictable behaviors.
278 \section sec-links Links
280 @see http://cr.yp.to/redo.html for the philosophy of <b>redo</b> and
281 https://github.com/apenwarr/redo for an implementation and some comprehensive documentation.
283 \section sec-licensing Licensing
285 @author Guillaume Melquiond
289 This program is free software: you can redistribute it and/or modify
290 it under the terms of the GNU General Public License as published by
291 the Free Software Foundation, either version 3 of the License, or
292 (at your option) any later version.
294 This program is distributed in the hope that it will be useful,
295 but WITHOUT ANY WARRANTY; without even the implied warranty of
296 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
297 GNU General Public License for more details.
301 #define WIN32_LEAN_AND_MEAN
320 #include <sys/stat.h>
321 #include <sys/types.h>
334 #include <winsock2.h>
336 typedef SOCKET socket_t;
338 #include <sys/socket.h>
340 #include <sys/wait.h>
341 typedef int socket_t;
342 enum { INVALID_SOCKET = -1 };
345 #if defined(WINDOWS) || defined(MACOSX)
346 enum { MSG_NOSIGNAL = 0 };
349 typedef std::list<std::string> string_list;
351 typedef std::set<std::string> string_set;
354 * Reference-counted shared object.
355 * @note The default constructor delays the creation of the object until it
356 * is first dereferenced.
366 content(T const &t): cnt(1), val(t) {}
368 mutable content *ptr;
369 ref_ptr(): ptr(NULL) {}
370 ref_ptr(T const &t): ptr(new content(t)) {}
371 ref_ptr(ref_ptr const &p): ptr(p.ptr) { if (ptr) ++ptr->cnt; }
372 ~ref_ptr() { if (ptr && --ptr->cnt == 0) delete ptr; }
373 ref_ptr &operator=(ref_ptr const &p)
375 if (ptr == p.ptr) return *this;
376 if (ptr && --ptr->cnt == 0) delete ptr;
383 if (!ptr) ptr = new content;
386 T *operator->() const { return &**this; }
395 typedef std::map<std::string, ref_ptr<dependency_t> > dependency_map;
397 typedef std::map<std::string, string_list> variable_map;
400 * Build status of a target.
404 Uptodate, ///< Target is up-to-date.
405 Todo, ///< Target is missing or obsolete.
406 Recheck, ///< Target has an obsolete dependency.
407 Running, ///< Target is being rebuilt.
408 Remade, ///< Target was successfully rebuilt.
409 Failed ///< Build failed for target.
413 * Build status of a target.
417 status_e status; ///< Actual status.
418 time_t last; ///< Last-modified date.
421 typedef std::map<std::string, status_t> status_map;
424 * A rule loaded from Remakefile.
428 string_list targets; ///< Files produced by this rule.
429 string_list deps; ///< Files used for an implicit call to remake at the start of the script.
430 std::string script; ///< Shell script for building the targets.
433 typedef std::list<rule_t> rule_list;
435 typedef std::map<std::string, ref_ptr<rule_t> > rule_map;
437 typedef std::map<int, string_list> job_targets_map;
439 typedef std::map<int, variable_map> job_variables_map;
441 typedef std::map<pid_t, int> pid_job_map;
444 * Client waiting for a request complete.
446 * There are two kinds of clients:
447 * - real clients, which are instances of remake created by built scripts,
448 * - pseudo clients, which are created by the server to build specific targets.
450 * Among pseudo clients, there are two categories:
451 * - original clients, which are created for the targets passed on the
452 * command line by the user or for the initial regeneration of the rule file,
453 * - dependency clients, which are created to handle rules that have
454 * explicit dependencies and thus to emulate a call to remake.
458 socket_t socket; ///< Socket used to reply to the client (invalid for pseudo clients).
459 int job_id; ///< Job for which the built script called remake and spawned the client (negative for original clients).
460 bool failed; ///< Whether some targets failed in mode -k.
461 string_list pending; ///< Targets not yet started.
462 string_set running; ///< Targets being built.
463 rule_t *delayed; ///< Rule that implicitly created a dependency client, and which script has to be started on request completion.
464 variable_map variables; ///< Variable that are passed on the command line of a remake invocation.
465 client_t(): socket(INVALID_SOCKET), job_id(-1), failed(false), delayed(NULL) {}
468 typedef std::list<client_t> client_list;
471 * Map from variable names to their content.
473 static variable_map variables;
476 * Precomputed variable assignments for shell usage.
478 static std::string variable_block;
481 * Map from targets to their known dependencies.
483 static dependency_map dependencies;
486 * Map from targets to their build status.
488 static status_map status;
491 * Set of generic rules loaded from Remakefile.
493 static rule_list generic_rules;
496 * Map from targets to specific rules loaded from Remakefile.
498 static rule_map specific_rules;
501 * Map from jobs to targets being built.
503 static job_targets_map job_targets;
506 * Map from jobs to job specific variables
508 static job_variables_map job_variables;
511 * Map from jobs to shell pids.
513 static pid_job_map job_pids;
516 * List of clients waiting for a request to complete.
517 * New clients are put to front, so that the build process is depth-first.
519 static client_list clients;
522 * Maximum number of parallel jobs (non-positive if unbounded).
523 * Can be modified by the -j option.
525 static int max_active_jobs = 1;
528 * Whether to keep building targets in case of failure.
529 * Can be modified by the -k option.
531 static bool keep_going = false;
534 * Number of jobs currently running:
535 * - it increases when a process is created in #run_script,
536 * - it decreases when a completion message is received in #finalize_job.
538 * @note There might be some jobs running while #clients is empty.
539 * Indeed, if a client requested two targets to be rebuilt, if they
540 * are running concurrently, if one of them fails, the client will
541 * get a failure notice and might terminate before the other target
544 static int running_jobs = 0;
547 * Number of jobs currently waiting for a build request to finish:
548 * - it increases when a build request is received in #accept_client
549 * (since the client is presumably waiting for the reply),
550 * - it decreases when a reply is sent in #complete_request.
552 static int waiting_jobs = 0;
555 * Global counter used to produce increasing job numbers.
558 static int job_counter = 0;
561 * Socket on which the server listens for client request.
563 static socket_t socket_fd;
566 * Whether the request of an original client failed.
568 static bool build_failure;
571 * Name of the server socket in the file system.
573 static char *socket_name;
576 * Name of the first target of the first specific rule, used for default run.
578 static std::string first_target;
581 * Whether a short message should be displayed for each target.
583 static bool show_targets = true;
586 * Whether script commands are echoed.
588 static bool echo_scripts = false;
590 static time_t now = time(NULL);
592 static std::string working_dir;
595 static volatile sig_atomic_t got_SIGCHLD = 0;
597 static void child_sig_handler(int)
607 log(): active(false), open(false), depth(0)
610 std::ostream &operator()()
612 if (open) std::cerr << std::endl;
614 std::cerr << std::string(depth * 2, ' ');
618 std::ostream &operator()(bool o)
620 if (o && open) std::cerr << std::endl;
623 if (o || !open) std::cerr << std::string(depth * 2, ' ');
632 struct log_auto_close
635 log_auto_close(): still_open(true)
640 if (debug.active && still_open) debug(false) << "done\n";
644 #define DEBUG if (debug.active) debug()
645 #define DEBUG_open log_auto_close auto_close; if (debug.active) debug(true)
646 #define DEBUG_close if ((auto_close.still_open = false), debug.active) debug(false)
649 * Return the original string if it does not contain any special characters,
650 * a quoted and escaped string otherwise.
652 static std::string escape_string(std::string const &s)
654 char const *quoted_char = ",: '";
655 char const *escaped_char = "\"\\$!";
656 bool need_quotes = false;
657 size_t len = s.length(), nb = len;
658 for (size_t i = 0; i < len; ++i)
660 if (strchr(quoted_char, s[i])) need_quotes = true;
661 if (strchr(escaped_char, s[i])) ++nb;
663 if (nb != len) need_quotes = true;
664 if (!need_quotes) return s;
665 std::string t(nb + 2, '\\');
667 for (size_t i = 0, j = 1; i < len; ++i, ++j)
669 if (strchr(escaped_char, s[i])) ++j;
677 * Initialize #working_dir.
679 void init_working_dir(const char* argv0)
683 char const *delim = "/\\";
687 if (!getenv("REMAKE_SOCKET"))
689 std::string path = argv0;
691 size_t found = path.find_last_of(delim);
692 if (found != std::string::npos)
694 path = path.substr(0, found);
695 std::cout << "Entering directory `" << path << "'" << std::endl;
696 if (chdir(path.c_str()))
698 perror("Failed to change working directory");
704 char *res = getcwd(buf, sizeof(buf));
707 perror("Failed to get working directory");
714 * Normalize an absolute path with respect to the working directory.
715 * Paths outside the working subtree are left unchanged.
717 static std::string normalize_abs(std::string const &s)
719 size_t l = working_dir.length();
720 if (s.compare(0, l, working_dir)) return s;
721 size_t ll = s.length();
722 if (ll == l) return ".";
725 size_t pos = s.rfind('/', l);
726 assert(pos != std::string::npos);
727 return s.substr(pos + 1);
729 if (ll == l + 1) return ".";
730 return s.substr(l + 1);
734 * Normalize a target name.
736 static std::string normalize(std::string const &s)
739 char const *delim = "/\\";
743 size_t prev = 0, len = s.length();
744 size_t pos = s.find_first_of(delim);
745 if (pos == std::string::npos) return s;
746 bool absolute = pos == 0;
752 std::string n = s.substr(prev, pos - prev);
755 if (!l.empty()) l.pop_back();
757 return normalize(working_dir + '/' + s);
763 if (pos >= len) break;
765 pos = s.find_first_of(delim, prev);
766 if (pos == std::string::npos) pos = len;
768 string_list::const_iterator i = l.begin(), i_end = l.end();
769 if (i == i_end) return absolute ? "/" : ".";
771 if (absolute) n.push_back('/');
773 for (++i; i != i_end; ++i)
778 if (absolute) return normalize_abs(n);
783 * Normalize the content of a list of targets.
785 static void normalize_list(string_list &l)
787 for (string_list::iterator i = l.begin(),
788 i_end = l.end(); i != i_end; ++i)
797 static void skip_spaces(std::istream &in)
800 while (strchr(" \t", (c = in.get()))) {}
801 if (in.good()) in.putback(c);
807 static void skip_eol(std::istream &in)
810 while (strchr("\r\n", (c = in.get()))) {}
811 if (in.good()) in.putback(c);
814 enum token_e { Word, Eol, Eof, Colon, Equal, Dollar, Rightpar, Comma };
817 * Skip spaces and return the kind of the next token.
819 static token_e next_token(std::istream &in)
825 if (!in.good()) return Eof;
828 case ':': return Colon;
829 case ',': return Comma;
830 case '=': return Equal;
831 case '$': return Dollar;
832 case ')': return Rightpar;
839 if (c != '\r' && c != '\n')
853 * Read a (possibly quoted) word.
855 static std::string read_word(std::istream &in)
859 if (!in.good()) return res;
860 char const *separators = " \t\r\n:$(),=\"";
861 bool quoted = c == '"';
864 if (strchr(separators, c))
874 if (!in.good()) return res;
886 if (strchr(separators, c))
896 static string_list read_words(std::istream &in);
899 * Execute a built-in function @a name and append its result to @a dest.
901 static void execute_function(std::istream &in, std::string const &name, string_list &dest)
906 std::cerr << "Failed to load rules: syntax error" << std::endl;
910 string_list fix = read_words(in);
911 if (next_token(in) != Comma) goto error;
913 string_list names = read_words(in);
914 if (next_token(in) != Rightpar) goto error;
916 size_t fixl = fix.size();
917 if (name == "addprefix")
919 for (string_list::const_iterator i = names.begin(),
920 i_end = names.end(); i != i_end; ++i)
927 string_list::const_iterator k = fix.begin();
928 for (size_t j = 1; j != fixl; ++j)
930 dest.push_back(*k++);
932 dest.push_back(*k++ + *i);
935 else if (name == "addsuffix")
937 for (string_list::const_iterator i = names.begin(),
938 i_end = names.end(); i != i_end; ++i)
945 string_list::const_iterator k = fix.begin();
946 dest.push_back(*i + *k++);
947 for (size_t j = 1; j != fixl; ++j)
949 dest.push_back(*k++);
957 * Read a list of words, possibly executing functions.
959 static string_list read_words(std::istream &in)
964 std::cerr << "Failed to load rules: syntax error" << std::endl;
970 switch (next_token(in))
973 res.push_back(read_word(in));
978 if (in.get() != '(') goto error;
979 std::string name = read_word(in);
980 if (name.empty()) goto error;
981 token_e tok = next_token(in);
985 variable_map::const_iterator i = variables.find(name);
986 if (i != variables.end())
987 res.insert(res.end(), i->second.begin(), i->second.end());
989 else execute_function(in, name, res);
999 * Serialize a variable map.
1001 std::string serialize_variables(variable_map &vars, char sep = ' ')
1003 std::ostringstream buf;
1004 for(variable_map::const_iterator i = vars.begin(),
1005 i_end = vars.end(); i != i_end; ++i)
1007 buf << i->first << '=';
1010 for (string_list::const_iterator j = i->second.begin(),
1011 j_end = i->second.end(); j != j_end; ++j)
1013 if (first) first = false;
1017 buf << escape_string(val) << sep;
1019 std::string s = buf.str();
1024 * Load dependencies from @a in.
1026 static void load_dependencies(std::istream &in)
1030 string_list targets = read_words(in);
1031 if (targets.empty()) return;
1032 DEBUG << "reading dependencies of target " << targets.front() << std::endl;
1033 if (in.get() != ':')
1035 std::cerr << "Failed to load database" << std::endl;
1038 ref_ptr<dependency_t> dep;
1039 dep->targets = targets;
1040 string_list d = read_words(in);
1041 dep->deps.insert(d.begin(), d.end());
1042 for (string_list::const_iterator i = targets.begin(),
1043 i_end = targets.end(); i != i_end; ++i)
1045 dependencies[*i] = dep;
1052 * Load known dependencies from file <tt>.remake</tt>.
1054 static void load_dependencies()
1056 DEBUG_open << "Loading database... ";
1057 std::ifstream in(".remake");
1060 DEBUG_close << "not found\n";
1063 load_dependencies(in);
1067 * Read a rule starting with target @a first, if nonempty.
1068 * Store into #generic_rules or #specific_rules depending on its genericity.
1070 static void load_rule(std::istream &in, std::string const &first)
1072 DEBUG_open << "Reading rule for target " << first << "... ";
1076 DEBUG_close << "failed\n";
1077 std::cerr << "Failed to load rules: syntax error" << std::endl;
1082 // Read targets and check genericity.
1083 string_list targets = read_words(in);
1084 if (!first.empty()) targets.push_front(first);
1085 else if (targets.empty()) goto error;
1086 else DEBUG << "actual target: " << targets.front() << std::endl;
1087 bool generic = false;
1088 normalize_list(targets);
1089 for (string_list::const_iterator i = targets.begin(),
1090 i_end = targets.end(); i != i_end; ++i)
1092 if (i->empty()) goto error;
1093 if ((i->find('%') != std::string::npos) != generic)
1095 if (i == targets.begin()) generic = true;
1099 std::swap(rule.targets, targets);
1101 if (in.get() != ':') goto error;
1103 // Read dependencies.
1104 rule.deps = read_words(in);
1105 normalize_list(rule.deps);
1108 if (c != '\r' && c != '\n') goto error;
1112 std::ostringstream buf;
1116 if (!in.good()) break;
1117 if (c == '\t' || c == ' ')
1119 in.get(*buf.rdbuf());
1120 if (in.fail() && !in.eof()) in.clear();
1122 else if (c == '\r' || c == '\n')
1130 rule.script = buf.str();
1132 // Add generic rules to the correct set.
1135 generic_rules.push_back(rule);
1139 // Rules with a nonempty script lump all their targets in the same
1140 // dependency set, while other rules behave as if they had been
1141 // replicated for each of their targets.
1142 if (!rule.script.empty())
1144 ref_ptr<dependency_t> dep;
1145 dep->targets = rule.targets;
1146 dep->deps.insert(rule.deps.begin(), rule.deps.end());
1147 for (string_list::const_iterator i = rule.targets.begin(),
1148 i_end = rule.targets.end(); i != i_end; ++i)
1150 ref_ptr<dependency_t> &d = dependencies[*i];
1151 dep->deps.insert(d->deps.begin(), d->deps.end());
1157 for (string_list::const_iterator i = rule.targets.begin(),
1158 i_end = rule.targets.end(); i != i_end; ++i)
1160 ref_ptr<dependency_t> &dep = dependencies[*i];
1161 if (dep->targets.empty()) dep->targets.push_back(*i);
1162 dep->deps.insert(rule.deps.begin(), rule.deps.end());
1166 if (first_target.empty())
1167 first_target = rule.targets.front();
1169 ref_ptr<rule_t> r(rule);
1170 for (string_list::const_iterator i = rule.targets.begin(),
1171 i_end = rule.targets.end(); i != i_end; ++i)
1173 std::pair<rule_map::iterator,bool> j =
1174 specific_rules.insert(std::make_pair(*i, r));
1175 if (j.second) continue;
1176 std::cerr << "Failed to load rules: " << *i
1177 << " cannot be the target of several rules" << std::endl;
1183 * Save all the dependencies in file <tt>.remake</tt>.
1185 static void save_dependencies()
1187 DEBUG_open << "Saving database... ";
1188 std::ofstream db(".remake");
1189 while (!dependencies.empty())
1191 ref_ptr<dependency_t> dep = dependencies.begin()->second;
1192 for (string_list::const_iterator i = dep->targets.begin(),
1193 i_end = dep->targets.end(); i != i_end; ++i)
1195 db << escape_string(*i) << ' ';
1196 dependencies.erase(*i);
1199 for (string_set::const_iterator i = dep->deps.begin(),
1200 i_end = dep->deps.end(); i != i_end; ++i)
1202 db << ' ' << escape_string(*i);
1210 * If some rules have dependencies and non-generic targets, add these
1211 * dependencies to the targets.
1213 static void load_rules()
1215 DEBUG_open << "Loading rules... ";
1219 std::cerr << "Failed to load rules: syntax error" << std::endl;
1222 std::ifstream in("Remakefile");
1225 std::cerr << "Failed to load rules: no Remakefile found" << std::endl;
1236 while (in.get() != '\n') {}
1240 if (c == ' ' || c == '\t') goto error;
1241 token_e tok = next_token(in);
1244 std::string name = read_word(in);
1245 if (name.empty()) goto error;
1246 if (next_token(in) == Equal)
1249 DEBUG << "Assignment to variable " << name << std::endl;
1250 variables[name] = read_words(in);
1253 else load_rule(in, name);
1255 else if (tok == Dollar)
1256 load_rule(in, std::string());
1260 // Generate script for variable assignment
1261 std::ostringstream buf;
1262 for (variable_map::const_iterator i = variables.begin(),
1263 i_end = variables.end(); i != i_end; ++i)
1265 std::ostringstream var;
1267 for (string_list::const_iterator j = i->second.begin(),
1268 j_end = i->second.end(); j != j_end; ++j)
1270 if (first) first = false;
1274 buf << i->first << '=' << escape_string(var.str()) << std::endl;
1276 variable_block = buf.str();
1280 * Substitute a pattern into a list of strings.
1282 static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
1284 for (string_list::const_iterator i = src.begin(),
1285 i_end = src.end(); i != i_end; ++i)
1287 size_t pos = i->find('%');
1288 if (pos == std::string::npos)dst.push_back(*i);
1289 else dst.push_back(i->substr(0, pos) + pat + i->substr(pos + 1));
1294 * Find a generic rule matching @a target:
1295 * - the one leading to shorter matches has priority,
1296 * - among equivalent rules, the earliest one has priority.
1298 static rule_t find_generic_rule(std::string const &target)
1300 size_t tlen = target.length(), plen = tlen + 1;
1302 for (rule_list::const_iterator i = generic_rules.begin(),
1303 i_end = generic_rules.end(); i != i_end; ++i)
1305 for (string_list::const_iterator j = i->targets.begin(),
1306 j_end = i->targets.end(); j != j_end; ++j)
1308 size_t len = j->length();
1309 if (tlen < len) continue;
1310 if (plen <= tlen - (len - 1)) continue;
1311 size_t pos = j->find('%');
1312 if (pos == std::string::npos) continue;
1313 size_t len2 = len - (pos + 1);
1314 if (j->compare(0, pos, target, 0, pos) ||
1315 j->compare(pos + 1, len2, target, tlen - len2, len2))
1317 plen = tlen - (len - 1);
1318 std::string pat = target.substr(pos, plen);
1320 rule.script = i->script;
1321 substitute_pattern(pat, i->targets, rule.targets);
1322 substitute_pattern(pat, i->deps, rule.deps);
1330 * Find a specific rule matching @a target. Return a generic one otherwise.
1331 * If there is both a specific rule with an empty script and a generic rule, the
1332 * generic one is returned after adding the dependencies of the specific one.
1334 static rule_t find_rule(std::string const &target)
1336 rule_map::const_iterator i = specific_rules.find(target),
1337 i_end = specific_rules.end();
1338 // If there is a specific rule with a script, return it.
1339 if (i != i_end && !i->second->script.empty()) return *i->second;
1340 rule_t grule = find_generic_rule(target);
1341 // If there is no generic rule, return the specific rule (no script), if any.
1342 if (grule.targets.empty())
1344 if (i != i_end) return *i->second;
1347 // Optimize the lookup when there is only one target (alread looked up).
1348 if (grule.targets.size() == 1)
1351 grule.deps.insert(grule.deps.end(),
1352 i->second->deps.begin(), i->second->deps.end());
1355 // Add the dependencies of the specific rules of every target to the
1356 // generic rule. If any of those rules has a nonempty script, error out.
1357 for (string_list::const_iterator j = grule.targets.begin(),
1358 j_end = grule.targets.end(); j != j_end; ++j)
1360 i = specific_rules.find(*j);
1361 if (i == i_end) continue;
1362 if (!i->second->script.empty()) return rule_t();
1363 grule.deps.insert(grule.deps.end(),
1364 i->second->deps.begin(), i->second->deps.end());
1370 * Compute and memoize the status of @a target:
1371 * - if the file does not exist, the target is obsolete,
1372 * - if any dependency is obsolete or younger than the file, it is obsolete,
1373 * - otherwise it is up-to-date.
1375 * @note With multiple targets, they all share the same status. (If one is
1376 * obsolete, they all are.) For the second rule above, the latest target
1377 * is chosen, not the oldest!
1379 static status_t const &get_status(std::string const &target)
1381 std::pair<status_map::iterator,bool> i =
1382 status.insert(std::make_pair(target, status_t()));
1383 status_t &ts = i.first->second;
1384 if (!i.second) return ts;
1385 DEBUG_open << "Checking status of " << target << "... ";
1386 dependency_map::const_iterator j = dependencies.find(target);
1387 if (j == dependencies.end())
1390 if (stat(target.c_str(), &s) != 0)
1392 DEBUG_close << "missing\n";
1397 DEBUG_close << "up-to-date\n";
1398 ts.status = Uptodate;
1399 ts.last = s.st_mtime;
1402 dependency_t const &dep = *j->second;
1403 status_e st = Uptodate;
1405 for (string_list::const_iterator k = dep.targets.begin(),
1406 k_end = dep.targets.end(); k != k_end; ++k)
1409 if (stat(k->c_str(), &s) != 0)
1411 if (st == Uptodate) DEBUG_close << *k << " missing\n";
1415 status[*k].last = s.st_mtime;
1416 if (s.st_mtime > latest) latest = s.st_mtime;
1418 if (st == Todo) goto update;
1419 for (string_set::const_iterator k = dep.deps.begin(),
1420 k_end = dep.deps.end(); k != k_end; ++k)
1422 status_t const &ts_ = get_status(*k);
1423 if (latest < ts_.last)
1425 DEBUG_close << "older than " << *k << std::endl;
1429 if (ts_.status == Uptodate) continue;
1431 DEBUG << "obsolete dependency " << *k << std::endl;
1434 if (st == Uptodate) DEBUG_close << "all siblings up-to-date\n";
1436 for (string_list::const_iterator k = dep.targets.begin(),
1437 k_end = dep.targets.end(); k != k_end; ++k)
1439 status[*k].status = st;
1445 * Change the status of @a target to #Remade or #Uptodate depending on whether
1446 * its modification time changed.
1448 static void update_status(std::string const &target)
1450 DEBUG_open << "Rechecking status of " << target << "... ";
1451 status_map::iterator i = status.find(target);
1452 assert (i != status.end());
1453 status_t &ts = i->second;
1457 DEBUG_close << "possibly remade\n";
1461 if (stat(target.c_str(), &s) != 0)
1463 DEBUG_close << "missing\n";
1466 else if (s.st_mtime != ts.last)
1468 DEBUG_close << "remade\n";
1469 ts.last = s.st_mtime;
1473 DEBUG_close << "unchanged\n";
1474 ts.status = Uptodate;
1479 * Check if all the prerequisites of @a target ended being up-to-date.
1481 static bool still_need_rebuild(std::string const &target)
1483 DEBUG_open << "Rechecking obsoleteness of " << target << "... ";
1484 status_map::const_iterator i = status.find(target);
1485 assert (i != status.end());
1486 if (i->second.status != Recheck) return true;
1487 dependency_map::const_iterator j = dependencies.find(target);
1488 assert(j != dependencies.end());
1489 dependency_t const &dep = *j->second;
1490 for (string_set::const_iterator k = dep.deps.begin(),
1491 k_end = dep.deps.end(); k != k_end; ++k)
1493 if (status[*k].status != Uptodate) return true;
1495 for (string_list::const_iterator k = dep.targets.begin(),
1496 k_end = dep.targets.end(); k != k_end; ++k)
1498 status[*k].status = Uptodate;
1500 DEBUG_close << "no longer obsolete\n";
1505 * Handle job completion.
1507 static void complete_job(int job_id, bool success)
1509 DEBUG_open << "Completing job " << job_id << "... ";
1510 job_targets_map::iterator i = job_targets.find(job_id);
1511 assert(i != job_targets.end());
1512 string_list const &targets = i->second;
1515 for (string_list::const_iterator j = targets.begin(),
1516 j_end = targets.end(); j != j_end; ++j)
1523 DEBUG_close << "failed\n";
1524 std::cerr << "Failed to build";
1525 for (string_list::const_iterator j = targets.begin(),
1526 j_end = targets.end(); j != j_end; ++j)
1528 status[*j].status = Failed;
1529 std::cerr << ' ' << *j;
1532 std::cerr << std::endl;
1534 job_targets.erase(i);
1538 * Execute the script from @a rule.
1540 static bool run_script(int job_id, rule_t const &rule)
1544 std::cout << "Building";
1545 for (string_list::const_iterator i = rule.targets.begin(),
1546 i_end = rule.targets.end(); i != i_end; ++i)
1548 std::cout << ' ' << *i;
1550 std::cout << std::endl;
1553 ref_ptr<dependency_t> dep;
1554 dep->targets = rule.targets;
1555 dep->deps.insert(rule.deps.begin(), rule.deps.end());
1556 for (string_list::const_iterator i = rule.targets.begin(),
1557 i_end = rule.targets.end(); i != i_end; ++i)
1559 dependencies[*i] = dep;
1562 DEBUG_open << "Starting script for job " << job_id << " with variables ("
1563 << (job_id >= 0 ? serialize_variables(job_variables[job_id], ' ') : "")
1570 CloseHandle(pfd[0]);
1571 CloseHandle(pfd[1]);
1573 DEBUG_close << "failed\n";
1574 complete_job(job_id, false);
1577 if (!CreatePipe(&pfd[0], &pfd[1], NULL, 0))
1579 if (!SetHandleInformation(pfd[0], HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT))
1582 ZeroMemory(&si, sizeof(STARTUPINFO));
1583 si.cb = sizeof(STARTUPINFO);
1584 si.hStdError = GetStdHandle(STD_ERROR_HANDLE);
1585 si.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
1586 si.hStdInput = pfd[0];
1587 si.dwFlags |= STARTF_USESTDHANDLES;
1588 PROCESS_INFORMATION pi;
1589 ZeroMemory(&pi, sizeof(PROCESS_INFORMATION));
1590 std::ostringstream buf;
1592 if (!SetEnvironmentVariable("REMAKE_JOB_ID", buf.str().c_str()))
1594 std::ostringstream argv;
1595 argv << "SH.EXE -e -s";
1596 if (echo_scripts) argv << " -v";
1597 for (string_list::const_iterator i = rule.targets.begin(),
1598 i_end = rule.targets.end(); i != i_end; ++i)
1600 argv << " \"" << escape_string(*i) << '"';
1602 if (!CreateProcess(NULL, (char *)argv.str().c_str(), NULL, NULL,
1603 true, 0, NULL, NULL, &si, &pi))
1607 CloseHandle(pi.hThread);
1608 std::string script = variable_block
1609 + (job_id >= 0 ? serialize_variables(job_variables[job_id], '\n') : "")
1611 DWORD len = script.length(), wlen;
1612 if (!WriteFile(pfd[1], script.c_str(), len, &wlen, NULL) || wlen < len)
1613 std::cerr << "Unexpected failure while sending script to shell" << std::endl;
1614 CloseHandle(pfd[0]);
1615 CloseHandle(pfd[1]);
1617 job_pids[pi.hProcess] = job_id;
1627 DEBUG_close << "failed\n";
1628 complete_job(job_id, false);
1631 if (pipe(pfd) == -1)
1633 if (pid_t pid = fork())
1635 if (pid == -1) goto error2;
1636 std::string script = variable_block
1637 + (job_id >= 0 ? serialize_variables(job_variables[job_id], '\n') : "")
1639 ssize_t len = script.length();
1640 if (write(pfd[1], script.c_str(), len) < len)
1641 std::cerr << "Unexpected failure while sending script to shell" << std::endl;
1645 job_pids[pid] = job_id;
1648 // Child process starts here.
1649 std::ostringstream buf;
1651 if (setenv("REMAKE_JOB_ID", buf.str().c_str(), 1))
1652 _exit(EXIT_FAILURE);
1653 int num = echo_scripts ? 4 : 3;
1654 char const **argv = new char const *[num + rule.targets.size() + 1];
1658 if (echo_scripts) argv[3] = "-v";
1659 for (string_list::const_iterator i = rule.targets.begin(),
1660 i_end = rule.targets.end(); i != i_end; ++i, ++num)
1662 argv[num] = i->c_str();
1671 execv("/bin/sh", (char **)argv);
1672 _exit(EXIT_FAILURE);
1677 * Create a job for @a target according to the loaded rules.
1678 * Mark all the targets from the rule as running and reset their dependencies.
1679 * If the rule has dependencies, create a new client to build them just
1680 * before @a current, and change @a current so that it points to it.
1682 static bool start(std::string const &target, client_list::iterator ¤t)
1684 DEBUG_open << "Starting job " << job_counter << " for " << target << "... ";
1685 rule_t rule = find_rule(target);
1686 if (rule.targets.empty())
1688 status[target].status = Failed;
1689 DEBUG_close << "failed\n";
1690 std::cerr << "No rule for building " << target << std::endl;
1693 for (string_list::const_iterator i = rule.targets.begin(),
1694 i_end = rule.targets.end(); i != i_end; ++i)
1696 status[*i].status = Running;
1698 int job_id = job_counter++;
1699 job_targets[job_id] = rule.targets;
1700 DEBUG << "Setting variables of job: " << job_id << " (spawn by: "
1701 << current->job_id << ") to "
1702 << serialize_variables(current->variables, ' ') << std::endl;
1703 job_variables[job_id] = current->variables;
1704 if (!rule.deps.empty())
1706 DEBUG << "Current client has job_id: " << job_id
1707 << " and variables " << serialize_variables(current->variables, ' ')
1709 client_t dep_client = client_t();
1710 dep_client.variables = current->variables;
1711 current = clients.insert(current, dep_client);
1712 current->job_id = job_id;
1713 current->pending = rule.deps;
1714 current->delayed = new rule_t(rule);
1717 return run_script(job_id, rule);
1721 * Send a reply to a client then remove it.
1722 * If the client was a dependency client, start the actual script.
1724 static void complete_request(client_t &client, bool success)
1726 DEBUG_open << "Completing request from client of job " << client.job_id << "... ";
1729 assert(client.socket == INVALID_SOCKET);
1732 if (still_need_rebuild(client.delayed->targets.front()))
1733 run_script(client.job_id, *client.delayed);
1734 else complete_job(client.job_id, true);
1736 else complete_job(client.job_id, false);
1737 delete client.delayed;
1739 else if (client.socket != INVALID_SOCKET)
1741 char res = success ? 1 : 0;
1742 send(client.socket, &res, 1, 0);
1744 closesocket(client.socket);
1746 close(client.socket);
1751 if (client.job_id < 0 && !success) build_failure = true;
1755 * Return whether there are slots for starting new jobs.
1757 static bool has_free_slots()
1759 if (max_active_jobs <= 0) return true;
1760 return running_jobs - waiting_jobs < max_active_jobs;
1764 * Update clients as long as there are free slots:
1765 * - check for running targets that have finished,
1766 * - start as many pending targets as allowed,
1767 * - complete the request if there are neither running nor pending targets
1768 * left or if any of them failed.
1770 static void update_clients()
1772 DEBUG_open << "Updating clients... ";
1773 for (client_list::iterator i = clients.begin(), i_next = i,
1774 i_end = clients.end(); i != i_end && has_free_slots(); i = i_next)
1777 DEBUG_open << "Handling client from job " << i->job_id << "... ";
1781 complete_request(*i, false);
1783 DEBUG_close << "failed\n";
1787 // Remove running targets that have finished.
1788 for (string_set::iterator j = i->running.begin(), j_next = j,
1789 j_end = i->running.end(); j != j_end; j = j_next)
1792 status_map::const_iterator k = status.find(*j);
1793 assert(k != status.end());
1794 switch (k->second.status)
1799 if (!keep_going) goto failed;
1804 i->running.erase(j);
1812 // Start pending targets.
1813 while (!i->pending.empty())
1815 std::string target = i->pending.front();
1816 i->pending.pop_front();
1817 switch (get_status(target).status)
1820 i->running.insert(target);
1824 if (!keep_going) goto failed;
1832 client_list::iterator j = i;
1833 if (!start(target, i)) goto pending_failed;
1834 j->running.insert(target);
1835 if (!has_free_slots()) return;
1836 // Job start might insert a dependency client.
1843 // Try to complete request.
1844 // (This might start a new job if it was a dependency client.)
1845 if (i->running.empty())
1847 if (i->failed) goto failed;
1848 complete_request(*i, true);
1850 DEBUG_close << "finished\n";
1856 * Create a named unix socket that listens for build requests. Also set
1857 * the REMAKE_SOCKET environment variable that will be inherited by all
1860 static void create_server()
1865 perror("Failed to create server");
1871 DEBUG_open << "Creating server... ";
1874 // Prepare a windows socket.
1875 struct sockaddr_in socket_addr;
1876 socket_addr.sin_family = AF_INET;
1877 socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
1878 socket_addr.sin_port = 0;
1880 // Create and listen to the socket.
1881 socket_fd = socket(AF_INET, SOCK_STREAM, 0);
1882 if (socket_fd < 0) goto error;
1883 if (!SetHandleInformation((HANDLE)socket_fd, HANDLE_FLAG_INHERIT, 0))
1885 if (bind(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
1887 int len = sizeof(sockaddr_in);
1888 if (getsockname(socket_fd, (struct sockaddr *)&socket_addr, &len))
1890 std::ostringstream buf;
1891 buf << socket_addr.sin_port;
1892 if (!SetEnvironmentVariable("REMAKE_SOCKET", buf.str().c_str()))
1894 if (listen(socket_fd, 1000)) goto error;
1896 // Set a handler for SIGCHLD then block the signal (unblocked during select).
1898 sigemptyset(&sigmask);
1899 sigaddset(&sigmask, SIGCHLD);
1900 if (sigprocmask(SIG_BLOCK, &sigmask, NULL) == -1) goto error;
1901 struct sigaction sa;
1903 sa.sa_handler = &child_sig_handler;
1904 sigemptyset(&sa.sa_mask);
1905 if (sigaction(SIGCHLD, &sa, NULL) == -1) goto error;
1907 // Prepare a named unix socket in temporary directory.
1908 socket_name = tempnam(NULL, "rmk-");
1909 if (!socket_name) goto error2;
1910 struct sockaddr_un socket_addr;
1911 size_t len = strlen(socket_name);
1912 if (len >= sizeof(socket_addr.sun_path) - 1) goto error2;
1913 socket_addr.sun_family = AF_UNIX;
1914 strcpy(socket_addr.sun_path, socket_name);
1915 len += sizeof(socket_addr.sun_family);
1916 if (setenv("REMAKE_SOCKET", socket_name, 1)) goto error;
1918 // Create and listen to the socket.
1920 socket_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
1921 if (socket_fd < 0) goto error;
1923 socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
1924 if (socket_fd < 0) goto error;
1925 if (fcntl(socket_fd, F_SETFD, FD_CLOEXEC) < 0) goto error;
1927 if (bind(socket_fd, (struct sockaddr *)&socket_addr, len))
1929 if (listen(socket_fd, 1000)) goto error;
1934 * Accept a connection from a client, get the job it spawned from,
1935 * get the targets, and mark them as dependencies of the job targets.
1937 void accept_client()
1939 DEBUG_open << "Handling client request... ";
1941 // Accept connection.
1943 socket_t fd = accept(socket_fd, NULL, NULL);
1944 if (fd == INVALID_SOCKET) return;
1945 if (!SetHandleInformation((HANDLE)fd, HANDLE_FLAG_INHERIT, 0))
1948 std::cerr << "Unexpected failure while setting connection with client" << std::endl;
1952 // WSAEventSelect puts sockets into nonblocking mode, so disable it here.
1954 if (ioctlsocket(fd, FIONBIO, &nbio)) goto error2;
1955 #elif defined(LINUX)
1956 int fd = accept4(socket_fd, NULL, NULL, SOCK_CLOEXEC);
1959 int fd = accept(socket_fd, NULL, NULL);
1961 if (fcntl(fd, F_SETFD, FD_CLOEXEC) < 0) return;
1963 clients.push_front(client_t());
1964 client_list::iterator proc = clients.begin();
1969 DEBUG_close << "failed\n";
1970 std::cerr << "Received an ill-formed client message" << std::endl;
1976 clients.erase(proc);
1980 // Receive message. Stop when encountering two nuls in a row.
1981 std::vector<char> buf;
1983 while (len < sizeof(int) + 2 || buf[len - 1] || buf[len - 2])
1985 buf.resize(len + 1024);
1986 ssize_t l = recv(fd, &buf[0] + len, 1024, 0);
1987 if (l <= 0) goto error;
1991 // Parse job that spawned the client.
1993 memcpy(&job_id, &buf[0], sizeof(int));
1995 proc->job_id = job_id;
1996 job_targets_map::const_iterator i = job_targets.find(job_id);
1997 if (i == job_targets.end()) goto error;
1998 DEBUG << "receiving request from job " << job_id << std::endl;
2000 // Parse the targets and mark them as dependencies from the job targets.
2001 dependency_t &dep = *dependencies[job_targets[job_id].front()];
2002 char const *p = &buf[0] + sizeof(int);
2006 if (len == 1 && p[0] == 1)
2008 //Finished parsing targets.
2013 std::string target(p, p + len);
2014 DEBUG << "adding dependency " << target << " to job " << job_id << std::endl;
2015 proc->pending.push_back(target);
2016 dep.deps.insert(target);
2023 if (len == 0) return;
2024 std::string line(p, p + len);
2025 std::istringstream in (line);
2026 std::string name = read_word(in);
2027 if (next_token(in) != Equal) {
2028 std::cerr << '\'' << line << "'" << std::endl;
2031 in.ignore(1); // ignore =.
2032 DEBUG << "adding variable " << line << " to job " << job_id << std::endl;
2033 string_list l = read_words(in);
2034 proc->variables[name] = l;
2042 * Loop until all the jobs have finished.
2049 if (running_jobs == 0)
2051 assert(clients.empty());
2054 DEBUG_open << "Handling events... ";
2056 size_t len = job_pids.size() + 1;
2059 for (pid_job_map::const_iterator i = job_pids.begin(),
2060 i_end = job_pids.end(); i != i_end; ++i, ++num)
2064 WSAEVENT aev = WSACreateEvent();
2066 WSAEventSelect(socket_fd, aev, FD_ACCEPT);
2067 DWORD w = WaitForMultipleObjects(len, h, false, INFINITE);
2068 WSAEventSelect(socket_fd, aev, 0);
2070 if (w < WAIT_OBJECT_0 || WAIT_OBJECT_0 + len <= w)
2072 if (w == WAIT_OBJECT_0 + len - 1)
2077 pid_t pid = h[w - WAIT_OBJECT_0];
2079 bool res = GetExitCodeProcess(pid, &s) && s == 0;
2081 pid_job_map::iterator i = job_pids.find(pid);
2082 assert(i != job_pids.end());
2083 int job_id = i->second;
2086 complete_job(job_id, res);
2089 sigemptyset(&emptymask);
2092 FD_SET(socket_fd, &fdset);
2093 int ret = pselect(socket_fd + 1, &fdset, NULL, NULL, NULL, &emptymask);
2094 if (ret > 0 /* && FD_ISSET(socket_fd, &fdset)*/) accept_client();
2095 if (!got_SIGCHLD) continue;
2099 while ((pid = waitpid(-1, &status, WNOHANG)) > 0)
2101 bool res = WIFEXITED(status) && WEXITSTATUS(status) == 0;
2102 pid_job_map::iterator i = job_pids.find(pid);
2103 assert(i != job_pids.end());
2104 int job_id = i->second;
2107 complete_job(job_id, res);
2114 * Load dependencies and rules, listen to client requests, and loop until
2115 * all the requests have completed.
2116 * If Remakefile is obsolete, perform a first run with it only, then reload
2117 * the rules, and perform a second with the original clients.
2119 void server_mode(string_list const &targets)
2121 load_dependencies();
2124 if (get_status("Remakefile").status != Uptodate)
2126 clients.push_back(client_t());
2127 clients.back().pending.push_back("Remakefile");
2129 if (build_failure) goto early_exit;
2131 specific_rules.clear();
2132 generic_rules.clear();
2133 first_target.clear();
2136 clients.push_back(client_t());
2137 if (!targets.empty()) clients.back().pending = targets;
2138 else if (!first_target.empty())
2139 clients.back().pending.push_back(first_target);
2143 remove(socket_name);
2144 save_dependencies();
2145 exit(build_failure ? EXIT_FAILURE : EXIT_SUCCESS);
2149 * Connect to the server @a socket_name, send a build request for @a targets,
2150 * and exit with the status returned by the server.
2152 void client_mode(char *socket_name, string_list const &targets)
2157 perror("Failed to send targets to server");
2160 if (targets.empty()) exit(EXIT_SUCCESS);
2161 DEBUG_open << "Connecting to server... ";
2163 // Connect to server.
2165 struct sockaddr_in socket_addr;
2166 socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2167 if (socket_fd < 0) goto error;
2168 socket_addr.sin_family = AF_INET;
2169 socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2170 socket_addr.sin_port = atoi(socket_name);
2171 if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2174 struct sockaddr_un socket_addr;
2175 size_t len = strlen(socket_name);
2176 if (len >= sizeof(socket_addr.sun_path) - 1) exit(EXIT_FAILURE);
2177 socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2178 if (socket_fd < 0) goto error;
2179 socket_addr.sun_family = AF_UNIX;
2180 strcpy(socket_addr.sun_path, socket_name);
2181 if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(socket_addr.sun_family) + len))
2185 if (setsockopt(socket_fd, SOL_SOCKET, SO_NOSIGPIPE, &set_option, sizeof(set_option)))
2190 // Send current job id.
2191 char *id = getenv("REMAKE_JOB_ID");
2192 int job_id = id ? atoi(id) : -1;
2193 if (send(socket_fd, (char *)&job_id, sizeof(job_id), MSG_NOSIGNAL) != sizeof(job_id))
2197 for (string_list::const_iterator i = targets.begin(),
2198 i_end = targets.end(); i != i_end; ++i)
2200 DEBUG_open << "Sending " << *i << "... ";
2201 ssize_t len = i->length() + 1;
2202 if (send(socket_fd, i->c_str(), len, MSG_NOSIGNAL) != len)
2205 // send 10 as as separator between targets and variables.
2207 if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
2209 if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
2211 // (maybe split vars in small chunks and send... seems like an overkil)
2212 std::string vars = serialize_variables(variables, 0);
2213 ssize_t sent = vars.size();
2214 DEBUG << "Sending variables: '" << vars << "' to the server" << std::endl;
2215 if (send(socket_fd, vars.data(), sent, MSG_NOSIGNAL) != sent)
2218 // Send terminating nul and wait for reply.
2219 if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
2220 if (recv(socket_fd, &result, 1, 0) != 1) exit(EXIT_FAILURE);
2221 exit(result ? EXIT_SUCCESS : EXIT_FAILURE);
2225 * Display usage and exit with @a exit_status.
2227 void usage(int exit_status)
2229 std::cerr << "Usage: remake [options] [target] ...\n"
2231 " -d Echo script commands.\n"
2232 " -d -d Print lots of debugging information.\n"
2233 " -h, --help Print this message and exit.\n"
2234 " -j[N], --jobs=[N] Allow N jobs at once; infinite jobs with no arg.\n"
2235 " -k Keep going when some targets cannot be made.\n"
2236 " -r Look up targets from the dependencies on standard input.\n"
2237 " -v V=X, --var V=X Initialize variable V with X"
2238 " -s, --silent, --quiet Do not echo targets.\n";
2243 * This program behaves in two different ways.
2245 * - If the environment contains the REMAKE_SOCKET variable, the client
2246 * connects to this socket and sends to the server its build targets.
2247 * It exits once it receives the server reply.
2249 * - Otherwise, it creates a server that waits for build requests. It
2250 * also creates a pseudo-client that requests the targets passed on the
2253 int main(int argc, char *argv[])
2255 init_working_dir(argv[0]);
2257 string_list targets;
2258 bool indirect_targets = false;
2260 // Parse command-line arguments.
2261 for (int i = 1; i < argc; ++i)
2263 std::string arg = argv[i];
2264 if (arg.empty()) usage(EXIT_FAILURE);
2265 if (arg == "-h" || arg == "--help") usage(EXIT_SUCCESS);
2267 if (echo_scripts) debug.active = true;
2268 else echo_scripts = true;
2269 else if (arg == "-k" || arg =="--keep-going")
2271 else if (arg == "-s" || arg == "--silent" || arg == "--quiet")
2272 show_targets = false;
2273 else if (arg == "-r")
2274 indirect_targets = true;
2275 else if (arg.compare(0, 2, "-j") == 0)
2276 max_active_jobs = atoi(arg.c_str() + 2);
2277 else if (arg.compare(0, 7, "--jobs=") == 0)
2278 max_active_jobs = atoi(arg.c_str() + 7);
2279 else if (arg == "-v" || arg == "--var")
2282 if (i == argc) usage(EXIT_FAILURE);
2284 std::istringstream in (arg);
2285 std::string name = read_word(in);
2286 if (next_token(in) != Equal) {
2287 std::cerr << "Invalid variable '" << arg << "'" << std::endl;
2291 string_list l = read_words(in);
2292 variables[name] = l;
2296 if (arg[0] == '-') usage(1);
2297 targets.push_back(normalize(arg));
2298 DEBUG << "New target: " << arg << '\n';
2302 if (indirect_targets)
2304 load_dependencies(std::cin);
2307 if (l.empty() && !dependencies.empty())
2309 l.push_back(dependencies.begin()->second->targets.front());
2311 for (string_list::const_iterator i = l.begin(),
2312 i_end = l.end(); i != i_end; ++i)
2314 dependency_map::const_iterator j = dependencies.find(*i);
2315 if (j == dependencies.end()) continue;
2316 dependency_t const &dep = *j->second;
2317 for (string_set::const_iterator k = dep.deps.begin(),
2318 k_end = dep.deps.end(); k != k_end; ++k)
2320 targets.push_back(normalize(*k));
2323 dependencies.clear();
2328 if (WSAStartup(MAKEWORD(2,2), &wsaData))
2330 std::cerr << "Unexpected failure while initializing Windows Socket" << std::endl;
2335 // Run as client if REMAKE_SOCKET is present in the environment.
2336 if (char *sn = getenv("REMAKE_SOCKET")) client_mode(sn, targets);
2338 // Otherwise run as server.
2339 server_mode(targets);