Chapter 2. Conventions and Patterns

This chapter will familiarize you with common conventions and patterns you will encounter with MakeKit. Although the build files you will write are just POSIX shell scripts, MakeKit adds a few twists to keep you on your toes:

Life in Shell

MakeKit uses POSIX shell as its implementation and build file language because it strikes a good balance between flexibility and simplicity. Some loftier build systems try to abstract the task of building source code into a platform-neutral, declarative syntax (I hear XML is popular) and eschew the quick-and-dirty direct execution of Makefiles and shell script. MakeKit also provides a bevy of modules and functions to keep the grittier details of turning source code into binaries out of sight and out of mind, but any project of non-trivial complexity must inevitably dabble in the arcane and ad-hoc. When it comes time to descend the ivory tower of abstraction you'll find awk, sed, and other UNIX workhorses close at hand.

For open source projects in particular, the choice of POSIX shell means fewer build dependencies for consumers of your source distribution. MakeKit comes with a copy of the excellent, fast-as-its-name-implies dash shell packed into a single .c file. The configure script will build it on the fly when first run, providing insurance against older platforms that lack POSIX-compliant /bin/sh.

Being able to unpack a source tarball onto any sane UNIX system (perhaps augmented with gcc) and build it without first installing the author's favorite build system or scripting language isn't an inconsequential feature. It is one of the reasons — outside of cultural inertia — for the continued ubiquity of GNU autotools. MakeKit achieves this without inscrutable preprocessing steps by perl and m4. According to my rough calculations that makes it 66% saner by volume.

Keyword Parameters

MakeKit provides a few amenities to make your stay in the world of shell script more enjoyable. As you may have noted in the previous chapter, many functions in MakeKit take parameters with explicit names of the form VAR=value. For example:

mk_program \
    PROGRAM="hello" \
    SOURCES="hello.c" \
    INCLUDEDIRS="../include" \
    LIBDEPS="foobar"

These keyword parameters make it clear at a glance what each one means even if you aren't familiar with the function in question. Note that because this is simply a call to a shell function, each line except the last needs a backslash to continue the parameter list on the next line. This is the recommended way to format calls to functions with many parameters.

The keyword parameters in a function call may appear in any order. Many functions have optional parameters that may be omitted entirely.

Important

Keyword parameters are intentionally evocative of the GNU automake syntax that many have grown accustomed to. However, don't forget that you are writing a shell script and not a Makefile!

You can write your own functions that take keyword parameters using mk_parse_params. This will set each keyword parameter passed to your function to the shell variable of the same name. You should consider using mk_push_vars and mk_pop_vars to avoid clobbering their existing values.

Quoting and Lists

Often a function will take a list of items as one of its keyword parameters, such as the SOURCES parameter in the above example. Lists in MakeKit are whitespace-separated. You could add goodbye.c to the above example as follows:

mk_program \
    PROGRAM="hello" \
    SOURCES="hello.c goodbye.c" \
    INCLUDEDIRS="../include" \
    LIBDEPS="foobar"

If you want to use spaces in an item, you will need to add an additional level of quoting around that item. Since the SOURCES parameter is enclosed in double quotes, the easiest option is to use single quotes within it:

mk_program \
    PROGRAM="hello" \
    SOURCES="hello.c goodbye.c 'this name has spaces.c'" \
    INCLUDEDIRS="../include" \
    LIBDEPS="foobar"

If a list is getting a bit long and you want to line wrap it, be sure to use a backslash at the end of each line to indicate that it continues to the next:

mk_program \
    PROGRAM="hello" \
    SOURCES="hello.c \
             goodbye.c \
             'this name has spaces.c'" \
    INCLUDEDIRS="../include" \
    LIBDEPS="foobar"

Any list passed in this manner must be internally shell-quoted to preserve characters that have special meaning to the shell, such as spaces and dollar signs. If the extra quoting is tripping you up, there is an alternative syntax that lets you specify a list as ordinary function parameters, removing one of the layers of quoting:

mk_program \
    PROGRAM="hello" \
    @SOURCES={ hello.c goodbye.c "this name has spaces.c" } \
    INCLUDEDIRS="../include" \
    LIBDEPS="foobar"

If you wish to programmatically quote a string or lists of strings so that they are suitable to pass as one of these parameters, use mk_quote or mk_quote_list. You can expand a quoted list into the position parameters ($1, $2, ...) using mk_unquote_list.

Special Variables

MakeKit allows you to declare variables with special behavior using the mk_declare function. There are 4 types of attributes which may be used in any combination:

Exported (mk_declare -e)

An exported variable will have its value at the time configure was run saved so that it is available when the user runs make. This is primarily useful when writing custom build rules.

Inherited (mk_declare -i)

An inherited variable has its value passed down from each MakeKitBuild to those specified in its SUBDIRS list. The value may be overridden by a subdirectory (and the new value will be passed to its subdirectories in turn), but it will not affect the value seen by sibling or parent directories.

You may prefer to think of inherited variables as being scoped according to the hierarchy of your MakeKitBuild files. Ordinary variables have completely dynamic scope, and any changes to them made by one build file will be seen by all others that are processed subsequently.

Output (mk_declare -o)

An output variable is one that can be substituted into a file processed by mk_output_file. This is equivalent to the variable substitution feature in GNU autoconf.

System (mk_declare -s)

A system variable can take on multiple values according to what system is currently being targeted by configure tests or build rules. For example, the MK_CC variable, which specifies the C compiler to use, might be different depending on whether we wish to build a program that will be run by the build system (the computer running MakeKit) or the host system (the computer that will run the final product). This distinction is important when cross-compiling.

When the current target system is changed with a function such as mk_system, values of system variables are swapped in and out appropriately.

File Layout

MakeKit allows you to organize your project in a hierarchy, dividing work among MakeKitBuild files in each subdirectory. This is leads to good separation of concerns, modularity, and composability. However, MakeKit always generates a single Makefile rooted firmly in the directory where configure was run, and all build rules are constructed relative to it. This avoids the many well-documented flaws of recursive make, but it means you must bear in mind following:

The Iron-clad Law of MakeKit

The current working directory when configuring or building is always MK_ROOT_DIR, the directory where configure was run.

Finding Your Files

Many MakeKit functions conveniently accept file paths relative to the current MakeKitBuild file in the hierarchy (in fact, as you will soon find out, it is a bit more sophisticated than that). However, if you are calling out to a trusty UNIX program such as sed or cp you will need to be a bit more specific about where to look. There are several variables that indicate areas where files of interest can be found:

Source directory (MK_SOURCE_DIR)

This is the top of your source tree where your top-level MakeKitBuild file can be found.

Source subdirectory (MK_SOURCE_SUBDIR)

This is the directory within your source tree where the MakeKitBuild file currently being processed can be found.

Object directory (MK_OBJECT_DIR)

This is the top of the object directory where intermediate build products are stored (in particular, .o files from the compiler). Its layout exactly mirrors that of your source tree.

Object subdirectory (MK_OBJECT_SUBDIR)

This is the subdirectory within the object tree that mirrors the current source subdirectory.

Stage directory (MK_STAGE_DIR)

This is the directory where final build products are "staged" before they are installed with make install or packaged up in your package format of choice. It has a layout mirroring that of the UNIX root filesystem: $MK_STAGE_DIR/usr/bin, $MK_STAGE_DIR/usr/lib, etc.

Target Notation

Rather than forcing you to use the aforementioned variables whenever you want to specify the location of a file, most MakeKit functions accept paths in a standard target notation.

A relative path (one not beginning with a /) indicates a file relative to MK_SOURCE_SUBDIR or MK_OBJECT_SUBDIR. If the file exists in the source directory at the time the target is resolved, then it is taken to be that file. Otherwise, it is assumed to be an intermediate build product in the object directory.

An absolute path (one beginning with /) indicates a file relative to MK_STAGE_DIR. For example, using mk_stage allows you to create a rule that simply copies a file. If you indicate that you want to copy a file to /etc/foo.conf, this will result in a Makefile rule that actually creates $MK_STAGE_DIR/etc/foo.conf.

Finally, there is a fully-qualified form that all targets are resolved to. If a target begins with @, it indicates that the following path should be interpreted verbatim. If the path is relative, it is relative to MK_ROOT_DIR. If it is absolute, then it indicates that precise absolute path on the filesystem. You can use mk_resolve_target to resolve any target to this canonical form.

In general, functions that perform configuration tests, process build files, or create build rules use target notation for any paths they accept. On the other hand, there are many pure utility functions (such as mk_mkdir) that perform useful filesystem operations or paper over incompatibilities between various flavors of UNIX; these accept ordinary paths.