2 @mainpage Remake, a build system that bridges the gap between make and redo.
4 As with <b>make</b>, <b>remake</b> uses a centralized rule file, which is
5 named <b>Remakefile</b>. It contains rules with a <em>make</em>-like
9 target1 target2 ... : dependency1 dependency2 ...
15 A target is known to be up-to-date if all its dependencies are. If it
16 has no known dependencies yet the file already exits, it is assumed to
17 be up-to-date. Obsolete targets are rebuilt thanks to the shell script
20 As with <b>redo</b>, <b>remake</b> supports dynamic dependencies in
21 addition to these static dependencies. Whenever a script executes
22 <tt>remake dependency4 dependency5 ...</tt>, these dependencies are
23 rebuilt if they are obsolete. (So <b>remake</b> acts like
24 <b>redo-ifchange</b>.) Moreover, these dependencies are stored in file
25 <b>.remake</b> so that they are remembered in subsequent runs. Note that
26 dynamic dependencies from previous runs are only used to decide whether a
27 target is obsolete; they are not automatically rebuilt when they are
28 obsolete yet a target depends on them. They will only be rebuilt once the
29 dynamic call to <b>remake</b> is executed.
31 In other words, the following two rules have almost the same behavior.
34 target1 target2 ... : dependency1 dependency2 ...
38 remake dependency1 dependency2 ...
42 (There is a difference if the targets already exist, have never been
43 built before, and the dependencies are either younger or obsolete, since
44 the targets will not be rebuilt in the second case.)
46 The above usage of dynamic dependencies is hardly useful. Their strength
47 lies in the fact that they can be computed on the fly:
51 gcc -MMD -MF $1.d -o $1 -c ${1%.o}.c
56 ocamldep ${1%.cmo}.ml | remake -r $1
57 ocamlc -c ${1%.cmo}.ml
59 after.xml: before.xml rules.xsl
60 xsltproc --load-trace -o after.xml rules.xsl before.xml 2> deps
61 remake $(sed -n -e "\\,//,! s,^.*URL=\"\\([^\"]*\\).*\$,\\1,p" deps)
65 Note that the first rule fails if any of the header files included by
66 a C source file has to be automatically generated. In that case, one
67 should perform a first call to <b>remake</b> them before calling the
68 compiler. (Dependencies from several calls to <b>remake</b> are
69 cumulative, so they will all be remembered the next time.)
71 \section sec-usage Usage
73 Usage: <tt>remake <i>options</i> <i>targets</i></tt>
77 - <tt>-d</tt>: Echo script commands.
78 - <tt>-j[N]</tt>, <tt>--jobs=[N]</tt>: Allow N jobs at once; infinite jobs
80 - <tt>-k</tt>, <tt>--keep-going</tt>: Keep going when some targets cannot be made.
81 - <tt>-r</tt>: Look up targets from the dependencies on standard input.
82 - <tt>-s</tt>, <tt>--silent</tt>, <tt>--quiet</tt>: Do not echo targets.
84 \section sec-syntax Syntax
86 Lines starting with a space character or a tabulation are assumed to be rule
87 scripts. They are only allowed after a rule header.
89 Lines starting with <tt>#</tt> are considered to be comments and are ignored.
90 They do interrupt rule scripts though.
92 Any other line is either a rule header or a variable definition. If such a
93 line ends with a backslash, the following line break is ignored and the line
94 extends to the next one.
96 Rule headers are a nonempty list of names, followed by a colon, followed by
97 another list of names, possibly empty. Variable definitions are a single
98 name followed by equal followed by a list of names, possibly empty. Basically,
99 the syntax of a rule is as follows:
102 targets : prerequisites
106 List of names are space-separated sequences of names. If a name contains a
107 space character, it should be put into double quotes. Names can not be any
108 of the following special characters <tt>:$(),="</tt>. Again, quotation
109 should be used. Quotation marks can be escaped by a backslash inside
112 \subsection sec-variables Variables
114 Variables can be used to factor lists of targets or dependencies. They are
115 expanded as they are encountered during <b>Remakefile</b> parsing.
123 Variables can be used inside rule scripts; they are available as non-exported
124 shell variables there.
126 \subsection sec-functions Built-in functions
128 <b>remake</b> also supports a few built-in functions inspired from <b>make</b>.
130 - <tt>$(addprefix <i>prefix</i>, <i>list</i>)</tt> returns the list obtained
131 by prepending its first argument to each element of its second argument.
132 - <tt>$(addsuffix <i>suffix</i>, <i>list</i>)</tt> returns the list obtained
133 by appending its first argument to each element of its second argument.
135 Note that functions are ignored inside scripts.
137 \section sec-semantics Semantics
139 \subsection src-obsolete When are targets obsolete?
141 A target is obsolete:
143 - if there is no file corresponding to the target, or to one of its siblings
144 in a multi-target rule,
145 - if any of its dynamic dependencies from a previous run or any of its static
146 prerequisites is obsolete,
147 - if the latest file corresponding to its siblings or itself is older than any
148 of its dynamic dependencies or static prerequisites.
150 In all the other cases, it is assumed to be up-to-date (and so are all its
151 siblings). Note that the last rule above says "latest" and not "earliest". While
152 it might cause some obsolete targets to go unnoticed in corner cases, it allows
153 for the following kind of rules:
156 config.h stamp-config_h: config.h.in config.status
157 ./config.status config.h
161 A <tt>config.status</tt> file generally does not update header files (here
162 <tt>config.h</tt>) if they would not change. As a consequence, if not for the
163 <tt>stamp-config_h</tt> file above, a header would always be considered obsolete
164 once one of its prerequisites is modified. Note that touching <tt>config.h</tt>
165 rather than <tt>stamp-config_h</tt> would defeat the point of not updating it
166 in the first place, since the program files would need to be rebuilt.
168 Once all the static prerequisites of a target have been rebuilt, <b>remake</b>
169 checks if the target still needs to be built. If it was obsolete only because
170 its dependencies needed to be rebuilt and none of them changed, the target is
171 assumed to be up-to-date.
173 \subsection sec-rules How are targets (re)built?
175 There are two kinds of rules. If any of the targets or prerequisites contains
176 a <tt>%</tt> character, the rule is said to be <em>generic</em>. All the
177 targets of the rule shall then contain a single <tt>%</tt> character. All the
178 other rules are said to be <em>specific</em>.
180 A rule is said to <em>match</em> a given target:
182 - if it is specific and the target appears inside its target list,
183 - if it is generic and there is a way to replace the <tt>%</tt> character
184 from one of its targets so that it matches the given target.
186 When <b>remake</b> tries to build a given target, it looks for a specific rule
187 that matches it. If there is one and its script is nonempty, it uses it to
190 Otherwise, it looks for a generic rule that match the target. If there are
191 several matching rules, it chooses the one with the shortest pattern (and if
192 there are several ones, the earliest one). <b>remake</b> then looks for
193 specific rules that match each target of the generic rule. All the
194 prerequisites of these specific rules are added to those of the generic rule.
195 The script of the generic rule is used to build the target.
201 commands building t%1 and t2%
204 commands building t2z
208 # t2x is built by the first rule (which also builds tx1) and its prerequisites are p1, px2
209 # t2y is built by the first rule (which also builds ty1) and its prerequisites are p1, py2, p3
210 # t2z is built by the second rule and its prerequisite is p4
213 The set of rules from <b>Remakefile</b> is ill-formed:
215 - if any specific rule matching a target of the generic rule has a nonempty script,
216 - if any target of the generic rule is matched by a generic rule with a shorter pattern.
218 \section sec-compilation Compilation
220 - On Linux, MacOSX, and BSD: <tt>g++ -o remake remake.cpp</tt>
221 - On Windows: <tt>g++ -o remake.exe remake.cpp -lws2_32</tt>
223 Installing <b>remake</b> is needed only if <b>Remakefile</b> does not
224 specify the path to the executable for its recursive calls. Thanks to its
225 single source file, <b>remake</b> can be shipped inside other packages and
226 built at configuration time.
228 \section sec-differences Differences with other build systems
230 Differences with <b>make</b>:
232 - Dynamic dependencies are supported.
233 - For rules with multiple targets, the shell script is executed only once
234 and is assumed to build all the targets. There is no need for
235 convoluted rules that are robust enough for parallel builds. For generic
236 rules, this is similar to the behavior of pattern rules from <b>gmake</b>.
237 - As with <b>redo</b>, only one shell is run when executing a script,
238 rather than one per script line. Note that the shells are run with
239 option <tt>-e</tt>, thus causing them to exit as soon as an error is
241 - The dependencies of generic rules (known as implicit rules in make lingo)
242 are not used to decide between several of them. <b>remake</b> does not
243 select one for which it could satisfy the dependencies.
244 - Variables and built-in functions are expanded as they are encountered
245 during <b>Remakefile</b> parsing.
247 Differences with <b>redo</b>:
249 - As with <b>make</b>, it is possible to write the following kind of rules
252 Remakefile: Remakefile.in ./config.status
253 ./config.status Remakefile
255 - If a target is already built the first time <b>remake</b> runs, it still
256 uses the static prerequisites of rules mentioning it to check whether it
257 needs to be rebuilt. It does not assume it to be up-to-date. As with
258 <b>redo</b> though, if its obsolete status would be due to a dynamic
259 dependency, it will go unnoticed; it should be removed beforehand.
260 - <b>remake</b> has almost no features: no checksum-based dependencies, no
261 compatibility with token servers, etc.
263 Differences with both <b>make</b> and <b>redo</b>:
265 - Multiple targets are supported.
266 - When executing shell scripts, positional variables <tt>$1</tt>,
267 <tt>$2</tt>, etc, point to the target names of the rule obtained after
268 substituting <tt>%</tt>. No other variables are defined.
270 \section sec-limitations Limitations
272 - When the user or a script calls <b>remake</b>, the current working
273 directory should be the one containing <b>Remakefile</b> (and thus
275 - Some cases of ill-formed rules are not caught by <b>remake</b> and can
276 thus lead to unpredictable behaviors.
278 \section sec-links Links
280 @see http://cr.yp.to/redo.html for the philosophy of <b>redo</b> and
281 https://github.com/apenwarr/redo for an implementation and some comprehensive documentation.
283 \section sec-licensing Licensing
285 @author Guillaume Melquiond
289 This program is free software: you can redistribute it and/or modify
290 it under the terms of the GNU General Public License as published by
291 the Free Software Foundation, either version 3 of the License, or
292 (at your option) any later version.
294 This program is distributed in the hope that it will be useful,
295 but WITHOUT ANY WARRANTY; without even the implied warranty of
296 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
297 GNU General Public License for more details.
301 #define WIN32_LEAN_AND_MEAN
320 #include <sys/stat.h>
321 #include <sys/types.h>
334 #include <winsock2.h>
336 typedef SOCKET socket_t;
338 #include <sys/socket.h>
340 #include <sys/wait.h>
341 typedef int socket_t;
342 enum { INVALID_SOCKET = -1 };
345 #if defined(WINDOWS) || defined(MACOSX)
346 enum { MSG_NOSIGNAL = 0 };
349 typedef std::list<std::string> string_list;
351 typedef std::set<std::string> string_set;
354 * Reference-counted shared object.
355 * @note The default constructor delays the creation of the object until it
356 * is first dereferenced.
366 content(T const &t): cnt(1), val(t) {}
368 mutable content *ptr;
369 ref_ptr(): ptr(NULL) {}
370 ref_ptr(T const &t): ptr(new content(t)) {}
371 ref_ptr(ref_ptr const &p): ptr(p.ptr) { if (ptr) ++ptr->cnt; }
372 ~ref_ptr() { if (ptr && --ptr->cnt == 0) delete ptr; }
373 ref_ptr &operator=(ref_ptr const &p)
375 if (ptr == p.ptr) return *this;
376 if (ptr && --ptr->cnt == 0) delete ptr;
383 if (!ptr) ptr = new content;
386 T *operator->() const { return &**this; }
395 typedef std::map<std::string, ref_ptr<dependency_t> > dependency_map;
397 typedef std::map<std::string, string_list> variable_map;
400 * Build status of a target.
404 Uptodate, ///< Target is up-to-date.
405 Todo, ///< Target is missing or obsolete.
406 Recheck, ///< Target has an obsolete dependency.
407 Running, ///< Target is being rebuilt.
408 Remade, ///< Target was successfully rebuilt.
409 Failed ///< Build failed for target.
413 * Build status of a target.
417 status_e status; ///< Actual status.
418 time_t last; ///< Last-modified date.
421 typedef std::map<std::string, status_t> status_map;
424 * A rule loaded from Remakefile.
428 string_list targets; ///< Files produced by this rule.
429 string_list deps; ///< Files used for an implicit call to remake at the start of the script.
430 std::string script; ///< Shell script for building the targets.
433 typedef std::list<rule_t> rule_list;
435 typedef std::map<std::string, ref_ptr<rule_t> > rule_map;
437 typedef std::map<int, string_list> job_targets_map;
439 typedef std::map<int, variable_map> job_variables_map;
441 typedef std::map<pid_t, int> pid_job_map;
444 * Client waiting for a request complete.
446 * There are two kinds of clients:
447 * - real clients, which are instances of remake created by built scripts,
448 * - pseudo clients, which are created by the server to build specific targets.
450 * Among pseudo clients, there are two categories:
451 * - original clients, which are created for the targets passed on the
452 * command line by the user or for the initial regeneration of the rule file,
453 * - dependency clients, which are created to handle rules that have
454 * explicit dependencies and thus to emulate a call to remake.
458 socket_t socket; ///< Socket used to reply to the client (invalid for pseudo clients).
459 int job_id; ///< Job for which the built script called remake and spawned the client (negative for original clients).
460 bool failed; ///< Whether some targets failed in mode -k.
461 string_list pending; ///< Targets not yet started.
462 string_set running; ///< Targets being built.
463 rule_t *delayed; ///< Rule that implicitly created a dependency client, and which script has to be started on request completion.
464 variable_map variables; ///< Variable that are passed on the command line of a remake invocation.
465 client_t(): socket(INVALID_SOCKET), job_id(-1), failed(false), delayed(NULL) {}
468 typedef std::list<client_t> client_list;
471 * Map from variable names to their content.
473 static variable_map variables;
476 * Precomputed variable assignments for shell usage.
478 static std::string variable_block;
481 * Map from targets to their known dependencies.
483 static dependency_map dependencies;
486 * Map from targets to their build status.
488 static status_map status;
491 * Set of generic rules loaded from Remakefile.
493 static rule_list generic_rules;
496 * Map from targets to specific rules loaded from Remakefile.
498 static rule_map specific_rules;
501 * Map from jobs to targets being built.
503 static job_targets_map job_targets;
506 * Map from jobs to job specific variables
508 static job_variables_map job_variables;
511 * Map from jobs to shell pids.
513 static pid_job_map job_pids;
516 * List of clients waiting for a request to complete.
517 * New clients are put to front, so that the build process is depth-first.
519 static client_list clients;
522 * Maximum number of parallel jobs (non-positive if unbounded).
523 * Can be modified by the -j option.
525 static int max_active_jobs = 1;
528 * Whether to keep building targets in case of failure.
529 * Can be modified by the -k option.
531 static bool keep_going = false;
534 * Number of jobs currently running:
535 * - it increases when a process is created in #run_script,
536 * - it decreases when a completion message is received in #finalize_job.
538 * @note There might be some jobs running while #clients is empty.
539 * Indeed, if a client requested two targets to be rebuilt, if they
540 * are running concurrently, if one of them fails, the client will
541 * get a failure notice and might terminate before the other target
544 static int running_jobs = 0;
547 * Number of jobs currently waiting for a build request to finish:
548 * - it increases when a build request is received in #accept_client
549 * (since the client is presumably waiting for the reply),
550 * - it decreases when a reply is sent in #complete_request.
552 static int waiting_jobs = 0;
555 * Global counter used to produce increasing job numbers.
558 static int job_counter = 0;
561 * Socket on which the server listens for client request.
563 static socket_t socket_fd;
566 * Whether the request of an original client failed.
568 static bool build_failure;
571 * Name of the server socket in the file system.
573 static char *socket_name;
576 * Name of the first target of the first specific rule, used for default run.
578 static std::string first_target;
581 * Whether a short message should be displayed for each target.
583 static bool show_targets = true;
586 * Whether script commands are echoed.
588 static bool echo_scripts = false;
590 static time_t now = time(NULL);
592 static std::string working_dir;
595 static volatile sig_atomic_t got_SIGCHLD = 0;
597 static void child_sig_handler(int)
607 log(): active(false), open(false), depth(0)
610 std::ostream &operator()()
612 if (open) std::cerr << std::endl;
614 std::cerr << std::string(depth * 2, ' ');
618 std::ostream &operator()(bool o)
620 if (o && open) std::cerr << std::endl;
623 if (o || !open) std::cerr << std::string(depth * 2, ' ');
632 struct log_auto_close
635 log_auto_close(): still_open(true)
640 if (debug.active && still_open) debug(false) << "done\n";
644 #define DEBUG if (debug.active) debug()
645 #define DEBUG_open log_auto_close auto_close; if (debug.active) debug(true)
646 #define DEBUG_close if ((auto_close.still_open = false), debug.active) debug(false)
649 * Return the original string if it does not contain any special characters,
650 * a quoted and escaped string otherwise.
652 static std::string escape_string(std::string const &s)
654 char const *quoted_char = ",: '";
655 char const *escaped_char = "\"\\$!";
656 bool need_quotes = false;
657 size_t len = s.length(), nb = len;
658 for (size_t i = 0; i < len; ++i)
660 if (strchr(quoted_char, s[i])) need_quotes = true;
661 if (strchr(escaped_char, s[i])) ++nb;
663 if (nb != len) need_quotes = true;
664 if (!need_quotes) return s;
665 std::string t(nb + 2, '\\');
667 for (size_t i = 0, j = 1; i < len; ++i, ++j)
669 if (strchr(escaped_char, s[i])) ++j;
677 * Initialize #working_dir.
679 void init_working_dir()
682 char *res = getcwd(buf, sizeof(buf));
685 perror("Failed to get working directory");
692 * Normalize an absolute path with respect to the working directory.
693 * Paths outside the working subtree are left unchanged.
695 static std::string normalize_abs(std::string const &s)
697 size_t l = working_dir.length();
698 if (s.compare(0, l, working_dir)) return s;
699 size_t ll = s.length();
700 if (ll == l) return ".";
703 size_t pos = s.rfind('/', l);
704 assert(pos != std::string::npos);
705 return s.substr(pos + 1);
707 if (ll == l + 1) return ".";
708 return s.substr(l + 1);
712 * Normalize a target name.
714 static std::string normalize(std::string const &s)
717 char const *delim = "/\\";
721 size_t prev = 0, len = s.length();
722 size_t pos = s.find_first_of(delim);
723 if (pos == std::string::npos) return s;
724 bool absolute = pos == 0;
730 std::string n = s.substr(prev, pos - prev);
733 if (!l.empty()) l.pop_back();
735 return normalize(working_dir + '/' + s);
741 if (pos >= len) break;
743 pos = s.find_first_of(delim, prev);
744 if (pos == std::string::npos) pos = len;
746 string_list::const_iterator i = l.begin(), i_end = l.end();
747 if (i == i_end) return absolute ? "/" : ".";
749 if (absolute) n.push_back('/');
751 for (++i; i != i_end; ++i)
756 if (absolute) return normalize_abs(n);
761 * Normalize the content of a list of targets.
763 static void normalize_list(string_list &l)
765 for (string_list::iterator i = l.begin(),
766 i_end = l.end(); i != i_end; ++i)
775 static void skip_spaces(std::istream &in)
778 while (strchr(" \t", (c = in.get()))) {}
779 if (in.good()) in.putback(c);
785 static void skip_eol(std::istream &in)
788 while (strchr("\r\n", (c = in.get()))) {}
789 if (in.good()) in.putback(c);
792 enum token_e { Word, Eol, Eof, Colon, Equal, Dollar, Rightpar, Comma };
795 * Skip spaces and return the kind of the next token.
797 static token_e next_token(std::istream &in)
803 if (!in.good()) return Eof;
806 case ':': return Colon;
807 case ',': return Comma;
808 case '=': return Equal;
809 case '$': return Dollar;
810 case ')': return Rightpar;
817 if (c != '\r' && c != '\n')
831 * Read a (possibly quoted) word.
833 static std::string read_word(std::istream &in)
837 if (!in.good()) return res;
838 char const *separators = " \t\r\n:$(),=\"";
839 bool quoted = c == '"';
842 if (strchr(separators, c))
852 if (!in.good()) return res;
864 if (strchr(separators, c))
874 static string_list read_words(std::istream &in);
877 * Execute a built-in function @a name and append its result to @a dest.
879 static void execute_function(std::istream &in, std::string const &name, string_list &dest)
884 std::cerr << "Failed to load rules: syntax error" << std::endl;
888 string_list fix = read_words(in);
889 if (next_token(in) != Comma) goto error;
891 string_list names = read_words(in);
892 if (next_token(in) != Rightpar) goto error;
894 size_t fixl = fix.size();
895 if (name == "addprefix")
897 for (string_list::const_iterator i = names.begin(),
898 i_end = names.end(); i != i_end; ++i)
905 string_list::const_iterator k = fix.begin();
906 for (size_t j = 1; j != fixl; ++j)
908 dest.push_back(*k++);
910 dest.push_back(*k++ + *i);
913 else if (name == "addsuffix")
915 for (string_list::const_iterator i = names.begin(),
916 i_end = names.end(); i != i_end; ++i)
923 string_list::const_iterator k = fix.begin();
924 dest.push_back(*i + *k++);
925 for (size_t j = 1; j != fixl; ++j)
927 dest.push_back(*k++);
935 * Read a list of words, possibly executing functions.
937 static string_list read_words(std::istream &in)
942 std::cerr << "Failed to load rules: syntax error" << std::endl;
948 switch (next_token(in))
951 res.push_back(read_word(in));
956 if (in.get() != '(') goto error;
957 std::string name = read_word(in);
958 if (name.empty()) goto error;
959 token_e tok = next_token(in);
963 variable_map::const_iterator i = variables.find(name);
964 if (i != variables.end())
965 res.insert(res.end(), i->second.begin(), i->second.end());
967 else execute_function(in, name, res);
977 * Serialize a variable map.
979 std::string serialize_variables(variable_map &vars, char sep = ' ')
981 std::ostringstream buf;
982 for(variable_map::const_iterator i = vars.begin(),
983 i_end = vars.end(); i != i_end; ++i)
985 buf << i->first << '=';
988 for (string_list::const_iterator j = i->second.begin(),
989 j_end = i->second.end(); j != j_end; ++j)
991 if (first) first = false;
995 buf << escape_string(val) << sep;
997 std::string s = buf.str();
1002 * Load dependencies from @a in.
1004 static void load_dependencies(std::istream &in)
1008 string_list targets = read_words(in);
1009 if (targets.empty()) return;
1010 DEBUG << "reading dependencies of target " << targets.front() << std::endl;
1011 if (in.get() != ':')
1013 std::cerr << "Failed to load database" << std::endl;
1016 ref_ptr<dependency_t> dep;
1017 dep->targets = targets;
1018 string_list d = read_words(in);
1019 dep->deps.insert(d.begin(), d.end());
1020 for (string_list::const_iterator i = targets.begin(),
1021 i_end = targets.end(); i != i_end; ++i)
1023 dependencies[*i] = dep;
1030 * Load known dependencies from file <tt>.remake</tt>.
1032 static void load_dependencies()
1034 DEBUG_open << "Loading database... ";
1035 std::ifstream in(".remake");
1038 DEBUG_close << "not found\n";
1041 load_dependencies(in);
1045 * Read a rule starting with target @a first, if nonempty.
1046 * Store into #generic_rules or #specific_rules depending on its genericity.
1048 static void load_rule(std::istream &in, std::string const &first)
1050 DEBUG_open << "Reading rule for target " << first << "... ";
1054 DEBUG_close << "failed\n";
1055 std::cerr << "Failed to load rules: syntax error" << std::endl;
1060 // Read targets and check genericity.
1061 string_list targets = read_words(in);
1062 if (!first.empty()) targets.push_front(first);
1063 else if (targets.empty()) goto error;
1064 else DEBUG << "actual target: " << targets.front() << std::endl;
1065 bool generic = false;
1066 normalize_list(targets);
1067 for (string_list::const_iterator i = targets.begin(),
1068 i_end = targets.end(); i != i_end; ++i)
1070 if (i->empty()) goto error;
1071 if ((i->find('%') != std::string::npos) != generic)
1073 if (i == targets.begin()) generic = true;
1077 std::swap(rule.targets, targets);
1079 if (in.get() != ':') goto error;
1081 // Read dependencies.
1082 rule.deps = read_words(in);
1083 normalize_list(rule.deps);
1086 if (c != '\r' && c != '\n') goto error;
1090 std::ostringstream buf;
1094 if (!in.good()) break;
1095 if (c == '\t' || c == ' ')
1097 in.get(*buf.rdbuf());
1098 if (in.fail() && !in.eof()) in.clear();
1100 else if (c == '\r' || c == '\n')
1108 rule.script = buf.str();
1110 // Add generic rules to the correct set.
1113 generic_rules.push_back(rule);
1117 // Rules with a nonempty script lump all their targets in the same
1118 // dependency set, while other rules behave as if they had been
1119 // replicated for each of their targets.
1120 if (!rule.script.empty())
1122 ref_ptr<dependency_t> dep;
1123 dep->targets = rule.targets;
1124 dep->deps.insert(rule.deps.begin(), rule.deps.end());
1125 for (string_list::const_iterator i = rule.targets.begin(),
1126 i_end = rule.targets.end(); i != i_end; ++i)
1128 ref_ptr<dependency_t> &d = dependencies[*i];
1129 dep->deps.insert(d->deps.begin(), d->deps.end());
1135 for (string_list::const_iterator i = rule.targets.begin(),
1136 i_end = rule.targets.end(); i != i_end; ++i)
1138 ref_ptr<dependency_t> &dep = dependencies[*i];
1139 if (dep->targets.empty()) dep->targets.push_back(*i);
1140 dep->deps.insert(rule.deps.begin(), rule.deps.end());
1144 if (first_target.empty())
1145 first_target = rule.targets.front();
1147 ref_ptr<rule_t> r(rule);
1148 for (string_list::const_iterator i = rule.targets.begin(),
1149 i_end = rule.targets.end(); i != i_end; ++i)
1151 std::pair<rule_map::iterator,bool> j =
1152 specific_rules.insert(std::make_pair(*i, r));
1153 if (j.second) continue;
1154 std::cerr << "Failed to load rules: " << *i
1155 << " cannot be the target of several rules" << std::endl;
1161 * Save all the dependencies in file <tt>.remake</tt>.
1163 static void save_dependencies()
1165 DEBUG_open << "Saving database... ";
1166 std::ofstream db(".remake");
1167 while (!dependencies.empty())
1169 ref_ptr<dependency_t> dep = dependencies.begin()->second;
1170 for (string_list::const_iterator i = dep->targets.begin(),
1171 i_end = dep->targets.end(); i != i_end; ++i)
1173 db << escape_string(*i) << ' ';
1174 dependencies.erase(*i);
1177 for (string_set::const_iterator i = dep->deps.begin(),
1178 i_end = dep->deps.end(); i != i_end; ++i)
1180 db << ' ' << escape_string(*i);
1188 * If some rules have dependencies and non-generic targets, add these
1189 * dependencies to the targets.
1191 static void load_rules()
1193 DEBUG_open << "Loading rules... ";
1197 std::cerr << "Failed to load rules: syntax error" << std::endl;
1200 std::ifstream in("Remakefile");
1203 std::cerr << "Failed to load rules: no Remakefile found" << std::endl;
1214 while (in.get() != '\n') {}
1218 if (c == ' ' || c == '\t') goto error;
1219 token_e tok = next_token(in);
1222 std::string name = read_word(in);
1223 if (name.empty()) goto error;
1224 if (next_token(in) == Equal)
1227 DEBUG << "Assignment to variable " << name << std::endl;
1228 variables[name] = read_words(in);
1231 else load_rule(in, name);
1233 else if (tok == Dollar)
1234 load_rule(in, std::string());
1238 // Generate script for variable assignment
1239 std::ostringstream buf;
1240 for (variable_map::const_iterator i = variables.begin(),
1241 i_end = variables.end(); i != i_end; ++i)
1243 std::ostringstream var;
1245 for (string_list::const_iterator j = i->second.begin(),
1246 j_end = i->second.end(); j != j_end; ++j)
1248 if (first) first = false;
1252 buf << i->first << '=' << escape_string(var.str()) << std::endl;
1254 variable_block = buf.str();
1258 * Substitute a pattern into a list of strings.
1260 static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
1262 for (string_list::const_iterator i = src.begin(),
1263 i_end = src.end(); i != i_end; ++i)
1265 size_t pos = i->find('%');
1266 if (pos == std::string::npos)dst.push_back(*i);
1267 else dst.push_back(i->substr(0, pos) + pat + i->substr(pos + 1));
1272 * Find a generic rule matching @a target:
1273 * - the one leading to shorter matches has priority,
1274 * - among equivalent rules, the earliest one has priority.
1276 static rule_t find_generic_rule(std::string const &target)
1278 size_t tlen = target.length(), plen = tlen + 1;
1280 for (rule_list::const_iterator i = generic_rules.begin(),
1281 i_end = generic_rules.end(); i != i_end; ++i)
1283 for (string_list::const_iterator j = i->targets.begin(),
1284 j_end = i->targets.end(); j != j_end; ++j)
1286 size_t len = j->length();
1287 if (tlen < len) continue;
1288 if (plen <= tlen - (len - 1)) continue;
1289 size_t pos = j->find('%');
1290 if (pos == std::string::npos) continue;
1291 size_t len2 = len - (pos + 1);
1292 if (j->compare(0, pos, target, 0, pos) ||
1293 j->compare(pos + 1, len2, target, tlen - len2, len2))
1295 plen = tlen - (len - 1);
1296 std::string pat = target.substr(pos, plen);
1298 rule.script = i->script;
1299 substitute_pattern(pat, i->targets, rule.targets);
1300 substitute_pattern(pat, i->deps, rule.deps);
1308 * Find a specific rule matching @a target. Return a generic one otherwise.
1309 * If there is both a specific rule with an empty script and a generic rule, the
1310 * generic one is returned after adding the dependencies of the specific one.
1312 static rule_t find_rule(std::string const &target)
1314 rule_map::const_iterator i = specific_rules.find(target),
1315 i_end = specific_rules.end();
1316 // If there is a specific rule with a script, return it.
1317 if (i != i_end && !i->second->script.empty()) return *i->second;
1318 rule_t grule = find_generic_rule(target);
1319 // If there is no generic rule, return the specific rule (no script), if any.
1320 if (grule.targets.empty())
1322 if (i != i_end) return *i->second;
1325 // Optimize the lookup when there is only one target (alread looked up).
1326 if (grule.targets.size() == 1)
1329 grule.deps.insert(grule.deps.end(),
1330 i->second->deps.begin(), i->second->deps.end());
1333 // Add the dependencies of the specific rules of every target to the
1334 // generic rule. If any of those rules has a nonempty script, error out.
1335 for (string_list::const_iterator j = grule.targets.begin(),
1336 j_end = grule.targets.end(); j != j_end; ++j)
1338 i = specific_rules.find(*j);
1339 if (i == i_end) continue;
1340 if (!i->second->script.empty()) return rule_t();
1341 grule.deps.insert(grule.deps.end(),
1342 i->second->deps.begin(), i->second->deps.end());
1348 * Compute and memoize the status of @a target:
1349 * - if the file does not exist, the target is obsolete,
1350 * - if any dependency is obsolete or younger than the file, it is obsolete,
1351 * - otherwise it is up-to-date.
1353 * @note With multiple targets, they all share the same status. (If one is
1354 * obsolete, they all are.) For the second rule above, the latest target
1355 * is chosen, not the oldest!
1357 static status_t const &get_status(std::string const &target)
1359 std::pair<status_map::iterator,bool> i =
1360 status.insert(std::make_pair(target, status_t()));
1361 status_t &ts = i.first->second;
1362 if (!i.second) return ts;
1363 DEBUG_open << "Checking status of " << target << "... ";
1364 dependency_map::const_iterator j = dependencies.find(target);
1365 if (j == dependencies.end())
1368 if (stat(target.c_str(), &s) != 0)
1370 DEBUG_close << "missing\n";
1375 DEBUG_close << "up-to-date\n";
1376 ts.status = Uptodate;
1377 ts.last = s.st_mtime;
1380 dependency_t const &dep = *j->second;
1381 status_e st = Uptodate;
1383 for (string_list::const_iterator k = dep.targets.begin(),
1384 k_end = dep.targets.end(); k != k_end; ++k)
1387 if (stat(k->c_str(), &s) != 0)
1389 if (st == Uptodate) DEBUG_close << *k << " missing\n";
1393 status[*k].last = s.st_mtime;
1394 if (s.st_mtime > latest) latest = s.st_mtime;
1396 if (st == Todo) goto update;
1397 for (string_set::const_iterator k = dep.deps.begin(),
1398 k_end = dep.deps.end(); k != k_end; ++k)
1400 status_t const &ts_ = get_status(*k);
1401 if (latest < ts_.last)
1403 DEBUG_close << "older than " << *k << std::endl;
1407 if (ts_.status == Uptodate) continue;
1409 DEBUG << "obsolete dependency " << *k << std::endl;
1412 if (st == Uptodate) DEBUG_close << "all siblings up-to-date\n";
1414 for (string_list::const_iterator k = dep.targets.begin(),
1415 k_end = dep.targets.end(); k != k_end; ++k)
1417 status[*k].status = st;
1423 * Change the status of @a target to #Remade or #Uptodate depending on whether
1424 * its modification time changed.
1426 static void update_status(std::string const &target)
1428 DEBUG_open << "Rechecking status of " << target << "... ";
1429 status_map::iterator i = status.find(target);
1430 assert (i != status.end());
1431 status_t &ts = i->second;
1435 DEBUG_close << "possibly remade\n";
1439 if (stat(target.c_str(), &s) != 0)
1441 DEBUG_close << "missing\n";
1444 else if (s.st_mtime != ts.last)
1446 DEBUG_close << "remade\n";
1447 ts.last = s.st_mtime;
1451 DEBUG_close << "unchanged\n";
1452 ts.status = Uptodate;
1457 * Check if all the prerequisites of @a target ended being up-to-date.
1459 static bool still_need_rebuild(std::string const &target)
1461 DEBUG_open << "Rechecking obsoleteness of " << target << "... ";
1462 status_map::const_iterator i = status.find(target);
1463 assert (i != status.end());
1464 if (i->second.status != Recheck) return true;
1465 dependency_map::const_iterator j = dependencies.find(target);
1466 assert(j != dependencies.end());
1467 dependency_t const &dep = *j->second;
1468 for (string_set::const_iterator k = dep.deps.begin(),
1469 k_end = dep.deps.end(); k != k_end; ++k)
1471 if (status[*k].status != Uptodate) return true;
1473 for (string_list::const_iterator k = dep.targets.begin(),
1474 k_end = dep.targets.end(); k != k_end; ++k)
1476 status[*k].status = Uptodate;
1478 DEBUG_close << "no longer obsolete\n";
1483 * Handle job completion.
1485 static void complete_job(int job_id, bool success)
1487 DEBUG_open << "Completing job " << job_id << "... ";
1488 job_targets_map::iterator i = job_targets.find(job_id);
1489 assert(i != job_targets.end());
1490 string_list const &targets = i->second;
1493 for (string_list::const_iterator j = targets.begin(),
1494 j_end = targets.end(); j != j_end; ++j)
1501 DEBUG_close << "failed\n";
1502 std::cerr << "Failed to build";
1503 for (string_list::const_iterator j = targets.begin(),
1504 j_end = targets.end(); j != j_end; ++j)
1506 status[*j].status = Failed;
1507 std::cerr << ' ' << *j;
1510 std::cerr << std::endl;
1512 job_targets.erase(i);
1516 * Execute the script from @a rule.
1518 static bool run_script(int job_id, rule_t const &rule)
1522 std::cout << "Building";
1523 for (string_list::const_iterator i = rule.targets.begin(),
1524 i_end = rule.targets.end(); i != i_end; ++i)
1526 std::cout << ' ' << *i;
1528 std::cout << std::endl;
1531 ref_ptr<dependency_t> dep;
1532 dep->targets = rule.targets;
1533 dep->deps.insert(rule.deps.begin(), rule.deps.end());
1534 for (string_list::const_iterator i = rule.targets.begin(),
1535 i_end = rule.targets.end(); i != i_end; ++i)
1537 dependencies[*i] = dep;
1540 DEBUG_open << "Starting script for job " << job_id << " with variables ("
1541 << (job_id >= 0 ? serialize_variables(job_variables[job_id], ' ') : "")
1548 CloseHandle(pfd[0]);
1549 CloseHandle(pfd[1]);
1551 DEBUG_close << "failed\n";
1552 complete_job(job_id, false);
1555 if (!CreatePipe(&pfd[0], &pfd[1], NULL, 0))
1557 if (!SetHandleInformation(pfd[0], HANDLE_FLAG_INHERIT, HANDLE_FLAG_INHERIT))
1560 ZeroMemory(&si, sizeof(STARTUPINFO));
1561 si.cb = sizeof(STARTUPINFO);
1562 si.hStdError = GetStdHandle(STD_ERROR_HANDLE);
1563 si.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
1564 si.hStdInput = pfd[0];
1565 si.dwFlags |= STARTF_USESTDHANDLES;
1566 PROCESS_INFORMATION pi;
1567 ZeroMemory(&pi, sizeof(PROCESS_INFORMATION));
1568 std::ostringstream buf;
1570 if (!SetEnvironmentVariable("REMAKE_JOB_ID", buf.str().c_str()))
1572 std::ostringstream argv;
1573 argv << "SH.EXE -e -s";
1574 if (echo_scripts) argv << " -v";
1575 for (string_list::const_iterator i = rule.targets.begin(),
1576 i_end = rule.targets.end(); i != i_end; ++i)
1578 argv << " \"" << escape_string(*i) << '"';
1580 if (!CreateProcess(NULL, (char *)argv.str().c_str(), NULL, NULL,
1581 true, 0, NULL, NULL, &si, &pi))
1585 CloseHandle(pi.hThread);
1586 std::string script = variable_block
1587 + (job_id >= 0 ? serialize_variables(job_variables[job_id], '\n') : "")
1589 DWORD len = script.length(), wlen;
1590 if (!WriteFile(pfd[1], script.c_str(), len, &wlen, NULL) || wlen < len)
1591 std::cerr << "Unexpected failure while sending script to shell" << std::endl;
1592 CloseHandle(pfd[0]);
1593 CloseHandle(pfd[1]);
1595 job_pids[pi.hProcess] = job_id;
1605 DEBUG_close << "failed\n";
1606 complete_job(job_id, false);
1609 if (pipe(pfd) == -1)
1611 if (pid_t pid = fork())
1613 if (pid == -1) goto error2;
1614 std::string script = variable_block
1615 + (job_id >= 0 ? serialize_variables(job_variables[job_id], '\n') : "")
1617 ssize_t len = script.length();
1618 if (write(pfd[1], script.c_str(), len) < len)
1619 std::cerr << "Unexpected failure while sending script to shell" << std::endl;
1623 job_pids[pid] = job_id;
1626 // Child process starts here.
1627 std::ostringstream buf;
1629 if (setenv("REMAKE_JOB_ID", buf.str().c_str(), 1))
1630 _exit(EXIT_FAILURE);
1631 int num = echo_scripts ? 4 : 3;
1632 char const **argv = new char const *[num + rule.targets.size() + 1];
1636 if (echo_scripts) argv[3] = "-v";
1637 for (string_list::const_iterator i = rule.targets.begin(),
1638 i_end = rule.targets.end(); i != i_end; ++i, ++num)
1640 argv[num] = i->c_str();
1649 execv("/bin/sh", (char **)argv);
1650 _exit(EXIT_FAILURE);
1655 * Create a job for @a target according to the loaded rules.
1656 * Mark all the targets from the rule as running and reset their dependencies.
1657 * If the rule has dependencies, create a new client to build them just
1658 * before @a current, and change @a current so that it points to it.
1660 static bool start(std::string const &target, client_list::iterator ¤t)
1662 DEBUG_open << "Starting job " << job_counter << " for " << target << "... ";
1663 rule_t rule = find_rule(target);
1664 if (rule.targets.empty())
1666 status[target].status = Failed;
1667 DEBUG_close << "failed\n";
1668 std::cerr << "No rule for building " << target << std::endl;
1671 for (string_list::const_iterator i = rule.targets.begin(),
1672 i_end = rule.targets.end(); i != i_end; ++i)
1674 status[*i].status = Running;
1676 int job_id = job_counter++;
1677 job_targets[job_id] = rule.targets;
1678 DEBUG << "Setting variables of job: " << job_id << " (spawn by: "
1679 << current->job_id << ") to "
1680 << serialize_variables(current->variables, ' ') << std::endl;
1681 job_variables[job_id] = current->variables;
1682 if (!rule.deps.empty())
1684 DEBUG << "Current client has job_id: " << job_id
1685 << " and variables " << serialize_variables(current->variables, ' ')
1687 client_t dep_client = client_t();
1688 dep_client.variables = current->variables;
1689 current = clients.insert(current, dep_client);
1690 current->job_id = job_id;
1691 current->pending = rule.deps;
1692 current->delayed = new rule_t(rule);
1695 return run_script(job_id, rule);
1699 * Send a reply to a client then remove it.
1700 * If the client was a dependency client, start the actual script.
1702 static void complete_request(client_t &client, bool success)
1704 DEBUG_open << "Completing request from client of job " << client.job_id << "... ";
1707 assert(client.socket == INVALID_SOCKET);
1710 if (still_need_rebuild(client.delayed->targets.front()))
1711 run_script(client.job_id, *client.delayed);
1712 else complete_job(client.job_id, true);
1714 else complete_job(client.job_id, false);
1715 delete client.delayed;
1717 else if (client.socket != INVALID_SOCKET)
1719 char res = success ? 1 : 0;
1720 send(client.socket, &res, 1, 0);
1722 closesocket(client.socket);
1724 close(client.socket);
1729 if (client.job_id < 0 && !success) build_failure = true;
1733 * Return whether there are slots for starting new jobs.
1735 static bool has_free_slots()
1737 if (max_active_jobs <= 0) return true;
1738 return running_jobs - waiting_jobs < max_active_jobs;
1742 * Update clients as long as there are free slots:
1743 * - check for running targets that have finished,
1744 * - start as many pending targets as allowed,
1745 * - complete the request if there are neither running nor pending targets
1746 * left or if any of them failed.
1748 static void update_clients()
1750 DEBUG_open << "Updating clients... ";
1751 for (client_list::iterator i = clients.begin(), i_next = i,
1752 i_end = clients.end(); i != i_end && has_free_slots(); i = i_next)
1755 DEBUG_open << "Handling client from job " << i->job_id << "... ";
1759 complete_request(*i, false);
1761 DEBUG_close << "failed\n";
1765 // Remove running targets that have finished.
1766 for (string_set::iterator j = i->running.begin(), j_next = j,
1767 j_end = i->running.end(); j != j_end; j = j_next)
1770 status_map::const_iterator k = status.find(*j);
1771 assert(k != status.end());
1772 switch (k->second.status)
1777 if (!keep_going) goto failed;
1782 i->running.erase(j);
1790 // Start pending targets.
1791 while (!i->pending.empty())
1793 std::string target = i->pending.front();
1794 i->pending.pop_front();
1795 switch (get_status(target).status)
1798 i->running.insert(target);
1802 if (!keep_going) goto failed;
1810 client_list::iterator j = i;
1811 if (!start(target, i)) goto pending_failed;
1812 j->running.insert(target);
1813 if (!has_free_slots()) return;
1814 // Job start might insert a dependency client.
1821 // Try to complete request.
1822 // (This might start a new job if it was a dependency client.)
1823 if (i->running.empty())
1825 if (i->failed) goto failed;
1826 complete_request(*i, true);
1828 DEBUG_close << "finished\n";
1834 * Create a named unix socket that listens for build requests. Also set
1835 * the REMAKE_SOCKET environment variable that will be inherited by all
1838 static void create_server()
1843 perror("Failed to create server");
1849 DEBUG_open << "Creating server... ";
1852 // Prepare a windows socket.
1853 struct sockaddr_in socket_addr;
1854 socket_addr.sin_family = AF_INET;
1855 socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
1856 socket_addr.sin_port = 0;
1858 // Create and listen to the socket.
1859 socket_fd = socket(AF_INET, SOCK_STREAM, 0);
1860 if (socket_fd < 0) goto error;
1861 if (!SetHandleInformation((HANDLE)socket_fd, HANDLE_FLAG_INHERIT, 0))
1863 if (bind(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
1865 int len = sizeof(sockaddr_in);
1866 if (getsockname(socket_fd, (struct sockaddr *)&socket_addr, &len))
1868 std::ostringstream buf;
1869 buf << socket_addr.sin_port;
1870 if (!SetEnvironmentVariable("REMAKE_SOCKET", buf.str().c_str()))
1872 if (listen(socket_fd, 1000)) goto error;
1874 // Set a handler for SIGCHLD then block the signal (unblocked during select).
1876 sigemptyset(&sigmask);
1877 sigaddset(&sigmask, SIGCHLD);
1878 if (sigprocmask(SIG_BLOCK, &sigmask, NULL) == -1) goto error;
1879 struct sigaction sa;
1881 sa.sa_handler = &child_sig_handler;
1882 sigemptyset(&sa.sa_mask);
1883 if (sigaction(SIGCHLD, &sa, NULL) == -1) goto error;
1885 // Prepare a named unix socket in temporary directory.
1886 socket_name = tempnam(NULL, "rmk-");
1887 if (!socket_name) goto error2;
1888 struct sockaddr_un socket_addr;
1889 size_t len = strlen(socket_name);
1890 if (len >= sizeof(socket_addr.sun_path) - 1) goto error2;
1891 socket_addr.sun_family = AF_UNIX;
1892 strcpy(socket_addr.sun_path, socket_name);
1893 len += sizeof(socket_addr.sun_family);
1894 if (setenv("REMAKE_SOCKET", socket_name, 1)) goto error;
1896 // Create and listen to the socket.
1898 socket_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
1899 if (socket_fd < 0) goto error;
1901 socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
1902 if (socket_fd < 0) goto error;
1903 if (fcntl(socket_fd, F_SETFD, FD_CLOEXEC) < 0) goto error;
1905 if (bind(socket_fd, (struct sockaddr *)&socket_addr, len))
1907 if (listen(socket_fd, 1000)) goto error;
1912 * Accept a connection from a client, get the job it spawned from,
1913 * get the targets, and mark them as dependencies of the job targets.
1915 void accept_client()
1917 DEBUG_open << "Handling client request... ";
1919 // Accept connection.
1921 socket_t fd = accept(socket_fd, NULL, NULL);
1922 if (fd == INVALID_SOCKET) return;
1923 if (!SetHandleInformation((HANDLE)fd, HANDLE_FLAG_INHERIT, 0))
1926 std::cerr << "Unexpected failure while setting connection with client" << std::endl;
1930 // WSAEventSelect puts sockets into nonblocking mode, so disable it here.
1932 if (ioctlsocket(fd, FIONBIO, &nbio)) goto error2;
1933 #elif defined(LINUX)
1934 int fd = accept4(socket_fd, NULL, NULL, SOCK_CLOEXEC);
1937 int fd = accept(socket_fd, NULL, NULL);
1939 if (fcntl(fd, F_SETFD, FD_CLOEXEC) < 0) return;
1941 clients.push_front(client_t());
1942 client_list::iterator proc = clients.begin();
1947 DEBUG_close << "failed\n";
1948 std::cerr << "Received an ill-formed client message" << std::endl;
1954 clients.erase(proc);
1958 // Receive message. Stop when encountering two nuls in a row.
1959 std::vector<char> buf;
1961 while (len < sizeof(int) + 2 || buf[len - 1] || buf[len - 2])
1963 buf.resize(len + 1024);
1964 ssize_t l = recv(fd, &buf[0] + len, 1024, 0);
1965 if (l <= 0) goto error;
1969 // Parse job that spawned the client.
1971 memcpy(&job_id, &buf[0], sizeof(int));
1973 proc->job_id = job_id;
1974 job_targets_map::const_iterator i = job_targets.find(job_id);
1975 if (i == job_targets.end()) goto error;
1976 DEBUG << "receiving request from job " << job_id << std::endl;
1978 // Parse the targets and mark them as dependencies from the job targets.
1979 dependency_t &dep = *dependencies[job_targets[job_id].front()];
1980 char const *p = &buf[0] + sizeof(int);
1984 if (len == 1 && p[0] == 1)
1986 //Finished parsing targets.
1991 std::string target(p, p + len);
1992 DEBUG << "adding dependency " << target << " to job " << job_id << std::endl;
1993 proc->pending.push_back(target);
1994 dep.deps.insert(target);
2001 if (len == 0) return;
2002 std::string line(p, p + len);
2003 std::istringstream in (line);
2004 std::string name = read_word(in);
2005 if (next_token(in) != Equal) {
2006 std::cerr << '\'' << line << "'" << std::endl;
2009 in.ignore(1); // ignore =.
2010 DEBUG << "adding variable " << line << " to job " << job_id << std::endl;
2011 string_list l = read_words(in);
2012 proc->variables[name] = l;
2020 * Loop until all the jobs have finished.
2027 if (running_jobs == 0)
2029 assert(clients.empty());
2032 DEBUG_open << "Handling events... ";
2034 size_t len = job_pids.size() + 1;
2037 for (pid_job_map::const_iterator i = job_pids.begin(),
2038 i_end = job_pids.end(); i != i_end; ++i, ++num)
2042 WSAEVENT aev = WSACreateEvent();
2044 WSAEventSelect(socket_fd, aev, FD_ACCEPT);
2045 DWORD w = WaitForMultipleObjects(len, h, false, INFINITE);
2046 WSAEventSelect(socket_fd, aev, 0);
2048 if (w < WAIT_OBJECT_0 || WAIT_OBJECT_0 + len <= w)
2050 if (w == WAIT_OBJECT_0 + len - 1)
2055 pid_t pid = h[w - WAIT_OBJECT_0];
2057 bool res = GetExitCodeProcess(pid, &s) && s == 0;
2059 pid_job_map::iterator i = job_pids.find(pid);
2060 assert(i != job_pids.end());
2061 int job_id = i->second;
2064 complete_job(job_id, res);
2067 sigemptyset(&emptymask);
2070 FD_SET(socket_fd, &fdset);
2071 int ret = pselect(socket_fd + 1, &fdset, NULL, NULL, NULL, &emptymask);
2072 if (ret > 0 /* && FD_ISSET(socket_fd, &fdset)*/) accept_client();
2073 if (!got_SIGCHLD) continue;
2077 while ((pid = waitpid(-1, &status, WNOHANG)) > 0)
2079 bool res = WIFEXITED(status) && WEXITSTATUS(status) == 0;
2080 pid_job_map::iterator i = job_pids.find(pid);
2081 assert(i != job_pids.end());
2082 int job_id = i->second;
2085 complete_job(job_id, res);
2092 * Load dependencies and rules, listen to client requests, and loop until
2093 * all the requests have completed.
2094 * If Remakefile is obsolete, perform a first run with it only, then reload
2095 * the rules, and perform a second with the original clients.
2097 void server_mode(string_list const &targets)
2099 load_dependencies();
2102 if (get_status("Remakefile").status != Uptodate)
2104 clients.push_back(client_t());
2105 clients.back().pending.push_back("Remakefile");
2107 if (build_failure) goto early_exit;
2109 specific_rules.clear();
2110 generic_rules.clear();
2111 first_target.clear();
2114 clients.push_back(client_t());
2115 if (!targets.empty()) clients.back().pending = targets;
2116 else if (!first_target.empty())
2117 clients.back().pending.push_back(first_target);
2121 remove(socket_name);
2122 save_dependencies();
2123 exit(build_failure ? EXIT_FAILURE : EXIT_SUCCESS);
2127 * Connect to the server @a socket_name, send a build request for @a targets,
2128 * and exit with the status returned by the server.
2130 void client_mode(char *socket_name, string_list const &targets)
2135 perror("Failed to send targets to server");
2138 if (targets.empty()) exit(EXIT_SUCCESS);
2139 DEBUG_open << "Connecting to server... ";
2141 // Connect to server.
2143 struct sockaddr_in socket_addr;
2144 socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2145 if (socket_fd < 0) goto error;
2146 socket_addr.sin_family = AF_INET;
2147 socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2148 socket_addr.sin_port = atoi(socket_name);
2149 if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2152 struct sockaddr_un socket_addr;
2153 size_t len = strlen(socket_name);
2154 if (len >= sizeof(socket_addr.sun_path) - 1) exit(EXIT_FAILURE);
2155 socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2156 if (socket_fd < 0) goto error;
2157 socket_addr.sun_family = AF_UNIX;
2158 strcpy(socket_addr.sun_path, socket_name);
2159 if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(socket_addr.sun_family) + len))
2163 if (setsockopt(socket_fd, SOL_SOCKET, SO_NOSIGPIPE, &set_option, sizeof(set_option)))
2168 // Send current job id.
2169 char *id = getenv("REMAKE_JOB_ID");
2170 int job_id = id ? atoi(id) : -1;
2171 if (send(socket_fd, (char *)&job_id, sizeof(job_id), MSG_NOSIGNAL) != sizeof(job_id))
2175 for (string_list::const_iterator i = targets.begin(),
2176 i_end = targets.end(); i != i_end; ++i)
2178 DEBUG_open << "Sending " << *i << "... ";
2179 ssize_t len = i->length() + 1;
2180 if (send(socket_fd, i->c_str(), len, MSG_NOSIGNAL) != len)
2183 // send 10 as as separator between targets and variables.
2185 if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
2187 if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
2189 // (maybe split vars in small chunks and send... seems like an overkil)
2190 std::string vars = serialize_variables(variables, 0);
2191 ssize_t sent = vars.size();
2192 DEBUG << "Sending variables: '" << vars << "' to the server" << std::endl;
2193 if (send(socket_fd, vars.data(), sent, MSG_NOSIGNAL) != sent)
2196 // Send terminating nul and wait for reply.
2197 if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
2198 if (recv(socket_fd, &result, 1, 0) != 1) exit(EXIT_FAILURE);
2199 exit(result ? EXIT_SUCCESS : EXIT_FAILURE);
2203 * Display usage and exit with @a exit_status.
2205 void usage(int exit_status)
2207 std::cerr << "Usage: remake [options] [target] ...\n"
2209 " -d Echo script commands.\n"
2210 " -d -d Print lots of debugging information.\n"
2211 " -h, --help Print this message and exit.\n"
2212 " -j[N], --jobs=[N] Allow N jobs at once; infinite jobs with no arg.\n"
2213 " -k Keep going when some targets cannot be made.\n"
2214 " -r Look up targets from the dependencies on standard input.\n"
2215 " -v V=X, --var V=X Initialize variable V with X"
2216 " -s, --silent, --quiet Do not echo targets.\n";
2221 * This program behaves in two different ways.
2223 * - If the environment contains the REMAKE_SOCKET variable, the client
2224 * connects to this socket and sends to the server its build targets.
2225 * It exits once it receives the server reply.
2227 * - Otherwise, it creates a server that waits for build requests. It
2228 * also creates a pseudo-client that requests the targets passed on the
2231 int main(int argc, char *argv[])
2235 string_list targets;
2236 bool indirect_targets = false;
2238 // Parse command-line arguments.
2239 for (int i = 1; i < argc; ++i)
2241 std::string arg = argv[i];
2242 if (arg.empty()) usage(EXIT_FAILURE);
2243 if (arg == "-h" || arg == "--help") usage(EXIT_SUCCESS);
2245 if (echo_scripts) debug.active = true;
2246 else echo_scripts = true;
2247 else if (arg == "-k" || arg =="--keep-going")
2249 else if (arg == "-s" || arg == "--silent" || arg == "--quiet")
2250 show_targets = false;
2251 else if (arg == "-r")
2252 indirect_targets = true;
2253 else if (arg.compare(0, 2, "-j") == 0)
2254 max_active_jobs = atoi(arg.c_str() + 2);
2255 else if (arg.compare(0, 7, "--jobs=") == 0)
2256 max_active_jobs = atoi(arg.c_str() + 7);
2257 else if (arg == "-v" || arg == "--var")
2260 if (i == argc) usage(EXIT_FAILURE);
2262 std::istringstream in (arg);
2263 std::string name = read_word(in);
2264 if (next_token(in) != Equal) {
2265 std::cerr << "Invalid variable '" << arg << "'" << std::endl;
2269 string_list l = read_words(in);
2270 variables[name] = l;
2274 if (arg[0] == '-') usage(1);
2275 targets.push_back(normalize(arg));
2276 DEBUG << "New target: " << arg << '\n';
2280 if (indirect_targets)
2282 load_dependencies(std::cin);
2285 if (l.empty() && !dependencies.empty())
2287 l.push_back(dependencies.begin()->second->targets.front());
2289 for (string_list::const_iterator i = l.begin(),
2290 i_end = l.end(); i != i_end; ++i)
2292 dependency_map::const_iterator j = dependencies.find(*i);
2293 if (j == dependencies.end()) continue;
2294 dependency_t const &dep = *j->second;
2295 for (string_set::const_iterator k = dep.deps.begin(),
2296 k_end = dep.deps.end(); k != k_end; ++k)
2298 targets.push_back(normalize(*k));
2301 dependencies.clear();
2306 if (WSAStartup(MAKEWORD(2,2), &wsaData))
2308 std::cerr << "Unexpected failure while initializing Windows Socket" << std::endl;
2313 // Run as client if REMAKE_SOCKET is present in the environment.
2314 if (char *sn = getenv("REMAKE_SOCKET")) client_mode(sn, targets);
2316 // Otherwise run as server.
2317 server_mode(targets);