Return to gnode.h CVS log

Up to [local] / src / usr.bin / make

Work done at p2k7.


This is a really big step towards getting parallel make to work.


Note that this is not yet complete. There are still a few `details' to
fix before this works 100%.  Specifically: sequential make (compat) and
parallel make don't use the same engine, and the parallel engine still
has a few limitations. For instance, some known issues:
- parallel make does not deal with .phony targets correctly all the time.
- some errors are deadly in parallel make mode.
- parallel make NEEDS way more sturdy correspondance of file system paths
and target names, since it often needs to match dependencies to targets
before the corresponding files exist.
- some local variables like $* get set in a bogus way in some cases.
- suffix handling has issues, especially related to the NULL suffix.
So, if you find stuff that does NOT yet work with parallel make, don't go
blindly try to fix the Makefile. It's very likely you might have stumbled
into a make bug. (unless you really, really, understand Makefiles, DON'T
GO CHANGING THEM YET).



Tested by lots of people, thanks go to miod@, and robert@ among other people.

Quick summary of what this does:

- remove `saving commands' extension (it's not really usable, nor used)
- move compat job runner and parallel interrupt handling into engine.c
- tweak the code so that both compat and parallel mode use the same job runner
and the same interrupt handling. Remove the other one.
- optimize job runner so that, in parallel mode, the last command does not
fork if we can avoid it (as it's already running in a sub shell).
- scrape all the code that dealt with creating shell scripts from commands.
- scrape all the code that dealt with recognizing special sequences in
command output to print/not print output.
- fix the parallel job pipe to not keep around file descriptors that are not
needed.
- replace the parallel job buffering with a nicer one, that deals with
non-blocking descriptors to try to agregate as much output from one job in
one go (greed) to unconfuse the users.
- create two pipes per job, so that stdout and stderr stay separate.
- make job token printing a debug option.
- always use the parallel job-runner to `execute' commands, even if we just
print them out.
- store list of errors encountered during parallel make running, and print them
on exit, so that we know what went wrong.
- add a dirty hack to targ.c to deal with paths produced by gccmakedep.

#ifndef GNODE_H
#define GNODE_H
/*	$OpenPackages$ */
/*	$OpenBSD: gnode.h,v 1.7 2007/11/02 17:27:24 espie Exp $ */

/*
 * Copyright (c) 2001 Marc Espie.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the
 *    documentation and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE OPENBSD PROJECT AND CONTRIBUTORS
 * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 * A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE OPENBSD
 * PROJECT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

#ifndef TIMESTAMP_TYPE
#include "timestamp_t.h"
#endif
#ifndef LIST_TYPE
#include "lst_t.h"
#endif
#ifndef SYMTABLE_H
#include "symtable.h"
#endif

struct Suff_;
/*-
 * The structure for an individual graph node. Each node has several
 * pieces of data associated with it.
 *	1) the name of the target it describes
 *	2) the location of the target file in the file system.
 *	3) the type of operator used to define its sources (qv. parse.c)
 *	4) whether it is involved in this invocation of make
 *	5) whether the target has been remade
 *	6) whether any of its children has been remade
 *	7) the number of its children that are, as yet, unmade
 *	8) its modification time
 *	9) the modification time of its youngest child (qv. make.c)
 *	10) a list of nodes for which this is a source
 *	11) a list of nodes on which this depends
 *	12) a list of nodes that depend on this, as gleaned from the
 *	    transformation rules.
 *	13) a list of nodes of the same name created by the :: operator
 *	14) a list of nodes that must be made (if they're made) before
 *	    this node can be, but that do no enter into the datedness of
 *	    this node.
 *	15) a list of nodes that must be made (if they're made) after
 *	    this node is, but that do not depend on this node, in the
 *	    normal sense.
 *	16) a Lst of ``local'' variables that are specific to this target
 *	   and this target only (qv. var.c [$@ $< $?, etc.])
 *	17) a Lst of strings that are commands to be given to a shell
 *	   to create this target.
 */

#define UNMADE		0
#define BEINGMADE	1
#define MADE		2
#define	UPTODATE	3
#define ERROR		4
#define ABORTED		5
#define CYCLE		6
#define ENDCYCLE	7
#define NOSUCHNODE	8

#define SPECIAL_NONE	0
#define	SPECIAL_PATH		21
#define SPECIAL_MASK		63
#define SPECIAL_TARGET		64
#define SPECIAL_SOURCE		128
#define SPECIAL_TARGETSOURCE	(SPECIAL_TARGET|SPECIAL_SOURCE)

struct GNode_ {
    int special_op;	/* special op to apply */
    unsigned char special;/* type of special node */
    char make;		/* true if this target needs to be remade */
    char childMade;	/* true if one of this target's children was
			 * made */
    char made;		/* Set to reflect the state of processing
			 * on this node:
			 *  UNMADE - Not examined yet
			 *  BEINGMADE - Target is already being made.
			 *	Indicates a cycle in the graph. (compat
			 *	mode only)
			 *  MADE - Was out-of-date and has been made
			 *  UPTODATE - Was already up-to-date
			 *  ERROR - An error occurred while it was being
			 *	made (used only in compat mode)
			 *  ABORTED - The target was aborted due to
			 *	an error making an inferior (compat).
			 *  CYCLE - Marked as potentially being part of
			 *	a graph cycle. If we come back to a
			 *	node marked this way, it is printed
			 *	and 'made' is changed to ENDCYCLE.
			 *  ENDCYCLE - the cycle has been completely
			 *	printed. Go back and unmark all its
			 *	members.
			 */
    char *path;		/* The full pathname of the file */
    int type;		/* Its type (see the OP flags, below) */
    int order;		/* Its wait weight */

    int unmade;		/* The number of unmade children */

    TIMESTAMP mtime;	/* Its modification time */
    TIMESTAMP cmtime;	/* The modification time of its youngest
			 * child */

    LIST iParents;	/* Links to parents for which this is an
			 * implied source, if any */
    LIST cohorts;	/* Other nodes for the :: operator */
    LIST parents;	/* Nodes that depend on this one */
    LIST children;	/* Nodes on which this one depends */
    LIST successors; 	/* Nodes that must be made after this one */
    LIST preds;		/* Nodes that must be made before this one */

    SymTable context;	/* The local variables */
    unsigned long lineno;/* First line number of commands.  */
    const char *fname;	/* File name of commands.  */
    LIST commands;	/* Creation commands */
    LstNode current;	/* Current command, for job */

    struct Suff_ *suffix;/* Suffix for the node (determined by
			 * Suff_FindDeps and opaque to everyone
			 * but the Suff module) */
    char name[1];	/* The target's name */
};

/*
 * The OP_ constants are used when parsing a dependency line as a way of
 * communicating to other parts of the program the way in which a target
 * should be made. These constants are bitwise-OR'ed together and
 * placed in the 'type' field of each node. Any node that has
 * a 'type' field which satisfies the OP_NOP function was never never on
 * the lefthand side of an operator, though it may have been on the
 * righthand side...
 */
#define OP_DEPENDS	0x00000001  /* Execution of commands depends on
				     * kids (:) */
#define OP_FORCE	0x00000002  /* Always execute commands (!) */
#define OP_DOUBLEDEP	0x00000004  /* Execution of commands depends on kids
				     * per line (::) */
#define OP_ERROR	0x00000000
#define OP_OPMASK	(OP_DEPENDS|OP_FORCE|OP_DOUBLEDEP)

#define OP_OPTIONAL	0x00000008  /* Don't care if the target doesn't
				     * exist and can't be created */
#define OP_USE		0x00000010  /* Use associated commands for parents */
#define OP_EXEC 	0x00000020  /* Target is never out of date, but always
				     * execute commands anyway. Its time
				     * doesn't matter, so it has none...sort
				     * of */
#define OP_IGNORE	0x00000040  /* Ignore errors when creating the node */
#define OP_PRECIOUS	0x00000080  /* Don't remove the target when
				     * interrupted */
#define OP_SILENT	0x00000100  /* Don't echo commands when executed */
#define OP_MAKE 	0x00000200  /* Target is a recursive make so its
				     * commands should always be executed when
				     * it is out of date, regardless of the
				     * state of the -n or -t flags */
#define OP_JOIN 	0x00000400  /* Target is out-of-date only if any of its
				     * children was out-of-date */
#define OP_MADE 	0x00000800  /* Assume the node is already made; even if
				     * it really is out of date */
#define OP_INVISIBLE	0x00004000  /* The node is invisible to its parents.
				     * I.e. it doesn't show up in the parents's
				     * local variables. */
#define OP_NOTMAIN	0x00008000  /* The node is exempt from normal 'main
				     * target' processing in parse.c */
#define OP_PHONY	0x00010000  /* Not a file target; run always */
#define OP_NOPATH	0x00020000  /* Don't search for file in the path */
#define OP_NODEFAULT	0x00040000  /* Special node that never needs */
				    /* DEFAULT commands applied */
#define OP_DUMMY	0x00080000  /* node was created by default, but it */
				    /* does not really exist. */
/* Attributes applied by PMake */
#define OP_TRANSFORM	0x80000000  /* The node is a transformation rule */
#define OP_MEMBER	0x40000000  /* Target is a member of an archive */
#define OP_LIB		0x20000000  /* Target is a library */
#define OP_ARCHV	0x10000000  /* Target is an archive construct */
#define OP_HAS_COMMANDS 0x08000000  /* Target has all the commands it should.
				     * Used when parsing to catch multiple
				     * commands for a target */
#define OP_DEPS_FOUND	0x02000000  /* Already processed by Suff_FindDeps */
#define OP_RESOLVED	0x01000000  /* We looked harder already */

/*
 * OP_NOP will return true if the node with the given type was not the
 * object of a dependency operator
 */
#define OP_NOP(t)	(((t) & OP_OPMASK) == 0x00000000)

#define OP_NOTARGET (OP_NOTMAIN|OP_USE|OP_EXEC|OP_TRANSFORM)


#endif