Make: Difference between revisions

From miki
Jump to navigation Jump to search
 
(45 intermediate revisions by the same user not shown)
Line 18: Line 18:
** Some even says [http://pozorvlak.livejournal.com/159126.html redo could be the new Git of build systems] (also [http://linuxfr.org/news/redo-un-remplaçant-de-choix-pour-make])
** Some even says [http://pozorvlak.livejournal.com/159126.html redo could be the new Git of build systems] (also [http://linuxfr.org/news/redo-un-remplaçant-de-choix-pour-make])
** ... but [http://synflood.at/blog/index.php?/archives/789-Why-djb-redo-wont-be-the-Git-of-build-systems.html some does not agree...]
** ... but [http://synflood.at/blog/index.php?/archives/789-Why-djb-redo-wont-be-the-Git-of-build-systems.html some does not agree...]
* [[Ninja]], a build system with '''focus on speed''', where scripts are intended to be generated by other tools (eg. cmake).


== Quick Reference ==
== Quick Reference ==
Line 38: Line 39:
|-
|-
|<tt>'''{{blue|$^}}'''</tt>
|<tt>'''{{blue|$^}}'''</tt>
|'''All prerequisites'''
|'''All prerequisites''', except order-only ones (see <code><nowiki>$|</nowiki></code>)
|-
|-
|<tt>'''{{blue|$+}}'''</tt>
|<tt>'''{{blue|$+}}'''</tt>
|'''All prerequisites, incl. duplicates'''
|'''All prerequisites, incl. duplicates'''
|-
|<tt>'''{{blue|1=<nowiki>$|</nowiki>}}'''</tt>
|'''All order-only prerequisites'''
|-
|-
|<tt>'''{{blue|$*}}'''</tt>
|<tt>'''{{blue|$*}}'''</tt>
Line 103: Line 107:
</source>
</source>


;Setting variables
=== Functions for File Names ===
Variables can be defined with <code>=</code> (''recursively expanded variable''), <code>:=</code> (''simply expanded variable''), <code>?=</code> (set if not set) or using keyword <code>override</code>.

This is illustrated in the script below.
<source lang=make>
ENV = ----
override ENVO = ----
ENVQ ?= ----
ARG = ----
override ARGO = ----
ARGQ ?= ----

all:
@echo " = over ?= # xxxx means variable was overridden."
@echo "ENV: $(ENV) $(ENVO) $(ENVQ)"
@echo "ARG: $(ARG) $(ARGO) $(ARGQ)"
@printf "%-20s %-20s %-20s\n" "$(origin ENV)" "$(origin ENVO)" "$(origin ENVQ)"
@printf "%-20s %-20s %-20s\n" "$(origin ARG)" "$(origin ARGO)" "$(origin ARGQ)"
</source>

This produces the following output:
<source lang=bash>
# ENV=xxxx ENVO=xxxx ENVQ=xxxx make ARG=xxxx ARGO=xxxx ARGQ=xxxx
# = over ?= # xxxx means variable was overridden.
# ENV: ---- ---- xxxx
# ARG: xxxx ---- xxxx
# file override environment
# command line override command line
</source>
Use flag <code>-e</code> to have environment variables to take precedence over file variables:
<source lang=bash>
ENV=xxxx ENVO=xxxx ENVQ=xxxx make -e ARG=xxxx ARGO=xxxx ARGQ=xxxx
# = over ?= # xxxx means variable was overridden.
# ENV: xxxx ---- xxxx
# ARG: xxxx ---- xxxx
# environment override override environment
# command line override command line
</source>

Note that <code>?=</code> is actually a shorthand notation:
<source lang=make>
FOO ?= bar

# is exactly equivalent to this (see section The origin Function):

ifeq ($(origin FOO), undefined)
FOO = bar
endif
</source>

So we can get the same for simply expanded variables:
<source lang=make>
ifeq ($(origin FOO), undefined)
FOO := bar # aka. FOO ?:= bar
endif
</source>

=== Functions for Filenames ===
;<tt>$(dir ''names...'')</tt>
;<tt>$(dir ''names...'')</tt>
:Extract the directory part
:Extract the directory part
Line 263: Line 324:
$(shell echo >&2 'ARM Based Posix system with J9DROP libraries')
$(shell echo >&2 'ARM Based Posix system with J9DROP libraries')
</source>
</source>
Note that <code>>&2</code> is necessary otherwise you get a strange error <code>missing separator</code> from make.


Alternatively one can use <code>info</code>:
Alternatively one can use <code>info</code>:
Line 360: Line 422:
bar:= $(subst $(space),$(comma),$(foo))
bar:= $(subst $(space),$(comma),$(foo))
# bar is now 'a,b,c'.
# bar is now 'a,b,c'.
# Note: slightly better is $(subst $(space),$(comma),$(strip $(foo)))
</source>
</source>


Line 425: Line 488:
</source>
</source>
Using <code>$(USAGE_TEXT)</code> would not work, or require escaping the end of line, add '\n', and use <code>echo -e</code>.
Using <code>$(USAGE_TEXT)</code> would not work, or require escaping the end of line, add '\n', and use <code>echo -e</code>.

Alternatively, for warning messages, we can do <code>$(warning $(USAGE_TEXT))</code>.


=== Passing target to make sub-module ===
=== Passing target to make sub-module ===
Line 455: Line 520:


=== Color messages ===
=== Color messages ===
<source lang=bash>
ESC1 := ^[
ESC2 := \033
ESC3 := $(shell echo "\033")


<source lang=bash>
# Some colors make a better life - The ^[ is a verbatim escape character
# We first fefine ESC - \033 doesn't work in $(info ...). The 2nd one is a verbatim escape char. The last one is most portable (dash and bash)
R := $(ESC1)[0;31m
# ... then we define the color code. DO NOT ADD TRAILING COMMENTS OR THIS WILL ADD TRAILING SPACES.
G := $(ESC2)[0;32m
# ESC := \033
B := $(ESC3)[0;34m")
Z := $(ESC1)[0;0m
# ESC := ^[
# ESC := $(shell echo "\033")
ESC := $(shell printf '\e')
R := $(ESC)[0;31m
G := $(ESC)[0;32m
B := $(ESC)[0;34m
Z := $(ESC)[0;0m


$(info A $Rcolorful $Zand $Gbeautiful $Bmessage$Z!)
$(info A $Rcolorful $Zand $Gbeautiful $Bmessage$Z!)
Line 471: Line 538:
</source>
</source>


Note that using <code>\033[0;32m</code> only works in shell <code>echo</code> command. The other two commands works in both <code>$(info ...)</code> and shell <code>echo</code> command.
Note that using <code>\033</code> only works in shell <code>echo</code> command. The other two commands works in both <code>$(info ...)</code> and shell <code>echo</code> command.

=== Color messages only if output to a tty ===
See my answer on [https://stackoverflow.com/questions/14625676/gnu-make-determine-if-stdout-is-terminal/62180185#62180185 StackOverflow]

;{{red|(new)}} Use MAKE_TERMOUT / MAKE_TERMERR variables (make 4.1+)

This is the best method if using make 4.1+.

<source lang="make">
# Simple solution
ifdef MAKE_TERMOUT
ifdef MAKE_TERMERR
# DO NOT ADD TRAILING COMMENTS OR THIS WILL FAIL BECAUSE OF TRAILING SPACES
ESC := $(shell printf '\e')
R := $(ESC)[31m
Z := $(ESC)[0;0m
endif
endif

$(info [info ] $Rred$Z black)
$(shell echo >&2 "[shell ] $Rred$Z black")
all:
@echo "[recipe] $Rred$Z black"
MAKE_TERMERR= $(MAKE) 2> /dev/null
MAKE_TERMOUT= $(MAKE) | cat

piped:
@echo "[recipe] $Rred$Z black"
</source>

;Detect TTY in a custom make script

We simply write a simple shell script that intercepts the call to the native make, detects the TTY, and defines a variable appropriately. The main advantages of this solution:
* simplicity,
* works for echo, $(info ...) and alike, $(shell ...) commands,
* no modification required in the recipes,
* no extra process spawned at each echo, which on some platform can be quite slow (eg. Cygwin).

<source lang="bash">
ifdef IS_TTY
# DO NOT ADD TRAILING COMMENTS OR THIS WILL FAIL BECAUSE OF TRAILING SPACES
ESC := $(shell printf '\e')
R := $(ESC)[31m
Z := $(ESC)[0;0m
endif

$(info [info ] $Rred$Z black)
$(shell echo >&2 "[shell ] $Rred$Z black")
all:
@echo "[recipe] $Rred$Z black"
</source>

Example of custom make script to add in path (eg. {{file|/usr/local/bin/make}}):

<source lang="bash">
#! /bin/bash

[ -t 1 ] && IS_TTY=1 || IS_TTY=0
exec /usr/bin/make IS_TTY=$IS_TTY "$@"
</source>

This method provides the expected output in all these cases:

<source lang="bash">
make # Colored
make >&2 # Colored
make | cat # NOT colored
make >tmp && cat tmp # NOT colored
</source>

; Detect in recipe and call the Makefile recursively

In some cases, it might not be possible to intercept the call to make or replace it with a custom script. In this solution, we fix that limitation by detecting the TTY in a first pass, and then calling the same Makefile recursively with the appropriate variable set.

<source lang="bash">
ifndef IS_TTY
.SILENT:
%:
@[ -t 1 ] && IS_TTY=1 || IS_TTY=0; $(MAKE) IS_TTY=$$IS_TTY "$@"
xyz:
@[ -t 1 ] && IS_TTY=1 || IS_TTY=0; $(MAKE) IS_TTY=$$IS_TTY
else

ifeq ($(IS_TTY),1)
# DO NOT ADD TRAILING COMMENTS OR THIS WILL FAIL BECAUSE OF TRAILING SPACES
ESC := $(shell printf '\e')
R := $(ESC)[31m
Z := $(ESC)[0;0m
endif

$(info [info ] $Rred$Z black)
$(shell echo >&2 "[shell ] $Rred$Z black")
recursive:
@echo "[recipe] $Rred$Z black"

endif
</source>

;Strip the escape sequences at each echo

We test at each echo command whether the output is a TTY, and we strip the escape sequences if not so (using https://superuser.com/questions/380772/removing-ansi-color-codes-from-text-stream). To reduce the spam on the line, we use a make variable $(STRIPESC). The solution is quite simple, but it only works for echo command, and spawns an extra process at each echo. It also requires to edit every recipe with an echo.

<source lang="bash">
# DO NOT ADD TRAILING COMMENTS OR THIS WILL FAIL BECAUSE OF TRAILING SPACES
ESC := $(shell printf '\e')
R := $(ESC)[31m
Z := $(ESC)[0;0m
STRIPESC:=( [ -t 1 ] && cat || sed 's/\x1b\[[0-9;]*m//g' )

# $(info ...) not supported
# $(shell echo >&2 ...) not supported
if_tty:
@echo "[recipe] $Rred$Z black" | $(STRIPESC)
</source>

;Use an external echo command to strip escape sequences

This is similar to the solution above except that the syntax is a bit more lightweight. The pros/cons are the same. Also, in the example below, I generate the external echo in the makefile itself. In a standard build, you would provide this command as an external tool in some standard path.

<source lang="bash">
# DO NOT ADD TRAILING COMMENTS OR THIS WILL FAIL BECAUSE OF TRAILING SPACES
ESC := $(shell printf '\e')
R := $(ESC)[31m
Z := $(ESC)[0;0m

$(shell echo "#! /bin/bash" > echotty)
$(shell echo '[ -t 1 ] && echo "$$@" || echo "$$@" | sed "s/\x1b\[[0-9;]*m//g"' >> echotty)
$(shell chmod a+x echotty)

# $(info ...)
# $(shell echo >&2 ...)
.PHONY: echotty
echotty:
@./echotty "[recipe] $Rred$Z black"
</source>

=== Detect if Make runs in an interactive shell ===
This one is simpler, and simply consists in testing whether FD 0 is a TTY [https://stackoverflow.com/questions/4251559/how-can-i-tell-if-a-makefile-is-being-run-from-an-interactive-shell]:

<source lang="bash">
INTERACTIVE:=$(shell [ -t 0 ] && echo 1)

ifdef INTERACTIVE
# is a terminal
else
# cron job
endif
</source>

=== Silent make by default, unless verbose requested ===
Make by default has ''echoing'', i.e. it prints recipe command before executing them. This can be silented by
* Prefixing command with <code>@</code>
* Using option <code>--silent</code> or <code>-s</code>
* Using special target <code>.SILENT</code>

A much better way is to disable echoing by default, and print relevant commands only in some case [http://stackoverflow.com/questions/24005166/gnu-make-silent-by-default]:

<source lang=make>
#Silent unless verbose (also for sub-make calls)
ifndef VERBOSE
.SILENT:
endif

# Same but syntax trick
$(VERBOSE).SILENT:

# Same but using MAKEFLAGS
ifndef VERBOSE
MAKEFLAGS += --silent
endif
</source>

Doing a <code>make VERBOSE=1</code> will print the recipe commands (unless if they are prefixed with <code>@</code>). All the tricks above also affect sub-make calls.

=== Subst a variable only if defined ===
Note that by default pattern substitution only occurs if the given variable is not empty:

<source lang=make>
$(shell touch foo.ld foo.c bar.c)

FOO=foo.ld

all: foo bar

foo: foo.c foo.c $(FOO)
@echo gcc $(FOO:%=-T %) $(filter %.c,$^) -o $@
@echo gcc $(patsubst %,-T %,$(filter %.ld,$^)) $(filter %.c,$^) -o $@

bar: bar.c $(BAR)
@echo gcc $(BAR:%=-T %) $(filter %.c,$^) -o $@
@echo gcc $(patsubst %,-T %,$(filter %.ld,$^)) $(filter %.c,$^) -o $@
</source>

This produces:

<source lang=text>
gcc -T foo.ld foo.c bar.c -o foo
gcc -T foo.ld foo.c bar.c -o foo
gcc bar.c -o bar
gcc bar.c -o bar
</source>

=== Enable or disable parallel build with MAKEFLAGS ===

Add flags <code>-j</code> to MAKEFLAGS. When there are many targets, it is better to limit a bit the number of parallel jobs (eg. <code>-j64</code>).

Be careful that not all Makefile are designed to run in parallel. Ideally, to prevent parallel execution, the {{file|Makefile}} must contain the <code>.NOTPARALLEL</code> target:

<source lang="makefile">
# This will prevent parallel execution of this Makefile
.NOTPARALLEL:
</source>

To preserve output consistency, we can also use <code>-O</code> to buffer output. For long build, <code>-O</code> often means there will be no output for a long time, so a better compromise is <code>-Oline</code>, which at least preserve output consistency for most parsers (like IDE that parses make output for errors).

<source lang="bash">
export MAKEFLAGS="-j64 -Oline" # 64 jobs, preserve line consistency
make
</source>

Alternatively we can add <code>-Onone</code> to <code>MAKEFLAGS</code>
<source lang="makefile">
# Make sure output-sync (-Oline) is disabled
MAKEFLAGS += -Onone
</source>

To disable parallel build for one target only:
<source lang="makefile">
# We don't want buffer when debugging
.PHONY: gdb gdb-nobuf
gdb: MAKEFLAGS+=-Onone
gdb: $(APP)
$(MAKE) gdb-nobuf

gdb-nobuf:
gdb $(APP)
</source>

Another method:
<source lang="makefile">
auto-my_target:
[ "$$MAKEFLAGS" != "-j1" ] && $(MAKE) $@ MAKEFLAGS=-j1 || $(@:auto-%=%)/gdb - -x gdbinit_auto
</source>

=== Use ccache to cache build objects ===
See [[ccache]].

=== Use target specific variable ===

One can define custom variables that are defined only for specific targets. The recipe syntax requires that '''variables be defined independently of other dependencies'''.

<source lang=make>
target: SOME_VAR = SOME_VALUE
target: src_a.c src_b.c
$(CC) $^ -o $@
</source>

Note that variables are defined on a '''separate line'''. To have the variable exported in the environment, use <code>export</code> keyword:

<source lang=make>
target: export SOME_VAR = SOME_VALUE
target: src_a.c src_b.c
$(CC) $^ -o $@
</source>

=== Disable implicit rules ===
Implicit rules are those that are defined by default in Make (like {{file|.c}} to {{file|.o}} conversion.

To disable them, just redefine <code>.SUFFIXES</code> variable:
<source lang=make>
.SUFFIXES:
</source>

=== Detect if running on Linux or Windows ===
We can use <code>OS</code> variable on Windows, but we cannot use <code>OSTYPE</code> on Linux (not defined in Makefile). So from [https://stackoverflow.com/questions/714100/os-detecting-makefile Stackoverflow], the source of all wisdom:
<source lang="make">
ifeq ($(OS),Windows_NT)
CCFLAGS += -D WIN32
ifeq ($(PROCESSOR_ARCHITEW6432),AMD64)
CCFLAGS += -D AMD64
else
ifeq ($(PROCESSOR_ARCHITECTURE),AMD64)
CCFLAGS += -D AMD64
endif
ifeq ($(PROCESSOR_ARCHITECTURE),x86)
CCFLAGS += -D IA32
endif
endif
else
UNAME_S := $(shell uname -s)
ifeq ($(UNAME_S),Linux)
CCFLAGS += -D LINUX
endif
ifeq ($(UNAME_S),Darwin)
CCFLAGS += -D OSX
endif
UNAME_P := $(shell uname -p)
ifeq ($(UNAME_P),x86_64)
CCFLAGS += -D AMD64
endif
ifneq ($(filter %86,$(UNAME_P)),)
CCFLAGS += -D IA32
endif
ifneq ($(filter arm%,$(UNAME_P)),)
CCFLAGS += -D ARM
endif
endif
</source>

=== Print newlines reliably in sh/bash/dash===
Use <code>printf</code> instead of <code>echo</code> to have more portable results [https://stackoverflow.com/questions/20172850/print-string-with-newline-reliably-in-both-bash-and-dash]:

<source lang="make">
all:
printf "hello\nworld\n" # Works for any SHELL
</source>

=== Use wildcard with file name with spaces ===
* [https://www.cmcrossroads.com/article/gnu-make-meets-file-names-spaces-them GNU Make meets file names spaces] (check link for more detailed analysis of impact of spaces)

<code>$(wildcard ...)</code> works with filenames with space if they are properly escaped:
<source lang="make">
empty:=
space:= $(empty) $(empty)
ifeq "$(wildcard $(subst $(space),\$(space),$(SOME_SPACED_PATH))/Include)" ""
$(info $(shell echo -e '$b$RERROR: PU path not found ($(SOME_SPACED_PATH)).$Z'))
$(error ERROR: path not found.)
endif
</source>

=== Strip trailing slash from directory ===
The simplest (also produce <code>.</code> if file has no directory part):
<source lang="make">
FILE:=foo/bar
PARENT:=$(patsubst %/,%,$(dir $(F)))
</source>

=== Force digest computation on up-to-date targets ===
We can use the double-colon rule to tell make to always redo some recipes, without triggering a complete rebuild (unlike <code>.PHONY</code> targets):
<source lang="make">
AXF:=$(DIR)/foo.axf
HEX:=$(DIR)/foo.hex
$(HEX):: $(AXF)
$(OBJCOPY) $(OBJCOPY_FLAGS) $@ $<

ifdef DIGEST
# According to manual, this double-colon rule w/o prerequisite is always executed, and always after the one on top...
$(HEX)::
sha1sum $@
endif
</source>


== Frequent Mistakes ==
== Frequent Mistakes ==
Line 486: Line 904:
;Use COMMA to separate parameters in $(call), not SPACE
;Use COMMA to separate parameters in $(call), not SPACE
:Do <code>$(call FCT,$(param))</code>, '''not''' <s><code>$(call FCT $(param))</code></s>. The later will fail ''silently''!
:Do <code>$(call FCT,$(param))</code>, '''not''' <s><code>$(call FCT $(param))</code></s>. The later will fail ''silently''!

;Use <code>$(shell echo ...)</code> outside of a recipe
:Using <code>$(shell echo ...)</code> in the body of the Makefile will generate an error <code>Missing separator</code>, because the output of the shell command is injected as is in the Makefile. Instead, either redirect with <code>$(shell echo &2 ...)</code>, or use <code>$(info $(shell echo ...))</code>, but oftentimes <code>$(shell echo )</code> is superfluous. [https://stackoverflow.com/questions/6510274/problem-with-executing-shell-script-from-makefile]

;MUST add space after <code>ifeq</code>, as in <code>ifeq ($OS,Windows_NT)</code>
:Otherwise make will complain with <code>*** missing separator. Stop.</code>


== Bugs ==
== Bugs ==

Latest revision as of 05:20, 11 September 2024

make is an utility that determines automatically which pieces of a program need to be recompiled, and issues the commands to rebuild them. make uses a file called Makefile, which describes the relationships among files in the program.

References

  • Recursive Make Considered Harmful
    This papers explains that cutting big projects into smaller Makefiles that are called recursively greatly increase the build time, even when nothing changed. Instead it advices to include smaller Makefiles in a master one, such that the dependency tree is complete.


Tools related to Make:

  • ccache, a compiler cache.
  • CMake, a cross-platform Make.

Some alternatives to Make:

Quick Reference

Automatic Variables

Make Automatic Variables (most frequent)
$@ Target of the rule (if multi target, the one that cause the rule's command to run)
$% Target member name, when target is an archive member
$< Name of 1st prerequisite
$? All newer prerequisites (more recent than the target)
$^ All prerequisites, except order-only ones (see $|)
$+ All prerequisites, incl. duplicates
$| All order-only prerequisites
$* The stem with which the rule matched (e.g. if target is dir/a.foo.b, a pattern like a.%.b gives a stem dir/foo)
Automatic Variables - F and D variants:
$(@D) $(@F) Resp. the directory (w/o trailing /) and filename part of the target
$(*D) $(*F) Resp. the directory (w/o trailing /) and filename part of the stem
$(<D) $(<F) Resp. the directory (w/o trailing /) and filename part of the 1st prerequisite
$(^D) $(^F) Resp. the directory (w/o trailing /) and filename part of the all prerequisite
$(+D) $(+F) Resp. the directory (w/o trailing /) and filename part of the all prerequisites, incl. duplicates
$(?D) $(?F) Resp. the directory (w/o trailing /) and filename part of the newer prerequisites

How to Use Variables

Recursive vs. simple expansion
# Recursive expansion
CFLAGS = $(INCLUDES) -O
INCLUDES = -Ifoo -Ibar

all:
	echo $(CFLAGS)           # will echo '-Ifoo -Ibar -O'
# Simple expansion
x := foo
y := $(x) bar                    # expansion occurs here, once for all (y = foo bar)
x := later                       # y still = foo bar
Substitution References
foo = a.o b.o c.o
bar = $(foo:.o=.c)               # bar = a.c b.c c.c
Using % (equiv. to $(patsubst %.c,%.o,$(foo)):
foo = a.o b.o c.o
bar = $(foo:%.o=%.c)             # bar = a.c b.c c.c
Setting variables

Variables can be defined with = (recursively expanded variable), := (simply expanded variable), ?= (set if not set) or using keyword override.

This is illustrated in the script below.

ENV = ----
override ENVO = ----
ENVQ ?= ----
ARG = ----
override ARGO = ----
ARGQ ?= ----

all:
	@echo "       =  over  ?=    # xxxx means variable was overridden."
	@echo "ENV: $(ENV) $(ENVO) $(ENVQ)"
	@echo "ARG: $(ARG) $(ARGO) $(ARGQ)" 
	@printf "%-20s %-20s %-20s\n" "$(origin ENV)" "$(origin ENVO)" "$(origin ENVQ)"
	@printf "%-20s %-20s %-20s\n" "$(origin ARG)" "$(origin ARGO)" "$(origin ARGQ)"

This produces the following output:

# ENV=xxxx ENVO=xxxx ENVQ=xxxx make ARG=xxxx ARGO=xxxx ARGQ=xxxx
#        =  over  ?=    # xxxx means variable was overridden.
# ENV: ---- ---- xxxx
# ARG: xxxx ---- xxxx
# file                 override             environment         
# command line         override             command line

Use flag -e to have environment variables to take precedence over file variables:

ENV=xxxx ENVO=xxxx ENVQ=xxxx make -e ARG=xxxx ARGO=xxxx ARGQ=xxxx
#        =  over  ?=    # xxxx means variable was overridden.
# ENV: xxxx ---- xxxx
# ARG: xxxx ---- xxxx
# environment override override             environment         
# command line         override             command line

Note that ?= is actually a shorthand notation:

FOO ?= bar

# is exactly equivalent to this (see section The origin Function):

ifeq ($(origin FOO), undefined)
FOO = bar
endif

So we can get the same for simply expanded variables:

ifeq ($(origin FOO), undefined)
FOO := bar          # aka. FOO ?:= bar
endif

Functions for Filenames

$(dir names...)
Extract the directory part
$(notdir names...)
Extract the not-directory part
$(suffix names...)
Extract the suffix of each file name (incl. the period .)
$(basename names...)
Extract the basename of each file name
$(addsuffix suffix,names...)
Append the value of suffix to each file name.
$(addsuffix .c,foo bar)        # Produces 'foo.c bar.c'
$(addprefix prefix,names...)
$(addprefix src/,foo bar)      # Produces 'src/foo src/bar'
$(join list1,list2)
$(wildcard pattern)
$(realpath names...)
$(abspath names...)

Rules

all: 
	@echo About to make distribution files    # @ : disable command echoing
	-rm $(file)                               # - : don't stop on error (ignore)

Tips

Automatic Prerequisites

A standard solution (inspired from Makefile manual):

# - see http://www.gnu.org/software/make/manual/make.html#Automatic-Prerequisites
# - Note that .d files are automatically regenerated because of the include command below.
# - Only 1 .d file generated, indep. of defines. So if include files depend on defines, the .d files will *NOT* be updated.
# - Note that we add *both* .o and .d files as targets (-MT flag). So any changes in the dependencies will regenerate both
#   .d and .o files.

-include $(SOURCES:.cpp=.d)

%.d: %.cpp
	$(CPP) $(CFLAGS) $(INCPATHS) -MM -MT $*.o -MT $*.d -MF $@ $<

A more advanced solution, where object files are stored in directory $(VARDIR), and have a suffix $(VARIANT):

# - see http://www.gnu.org/software/make/manual/make.html#Automatic-Prerequisites
# - Note that .d files are automatically regenerated because of the include command below.
# - Only 1 .d file generated, indep. of defines. So if include files depend on defines, the .d files will *NOT* be updated.
# - [CHANGES] .o and .d files are stored in directory $(VARDIR)/, and .o have suffix $(VARIANT)

-include $(addprefix $(VARDIR)/, $(notdir $(SOURCES:.cpp=.d) ) )

$(VARDIR)/%.d: $(SRCDIR)/%.cpp
	$(CPP) $(CFLAGS) $(INCPATHS) -MM -MT $(VARDIR)/$*$$"(VARIANT)".o -MT $*.d -MF $@ $<


Limitations and Other solutions:

  • Only one .d file generated, indep. of $(DEFINES), so if $(DEFINES) changed, and include files depends on these defines, the .d files will not be updated.
A solution would be to name the .d files according to relevant defines, as we do for the .o files
  • Instead of adding both .d and .o files to the target, one could leave the .d file as the only target, and add .d file as an extra dependency to the .o file.


Yet another solution, where .d are generated along with .o files:

SOURCES = OBJECTS = $(SOURCES:.c:.o)

ifeq "$(notdir $(CC)" "armcc"
# armcc will generate windows path. We use sed to fix the paths. Unfiltered deps are written to .ud files
# so that these are never included even if makefile is interrupted right after $(CC) (e.g. compilation errors)
%.o: %.c Makefile
      $(CC) $(CFLAGS) --unix_depend_format --depend $(@:.o=.ud) -c $< -o $@
      @sed -r 's!\\!/!g; s!([a-zA-Z]):/!/cygdrive/\1/!g' $(@:.o=.ud) > $(@:.o=.d)
else
# assuming 'gcc'
%.o: %.c Makefile
      $(CC) $(CFLAGS) -MMD -c $< -o $@
endif

# Create dependency file names from object names
DEPFILES = $(OBJECTS:.o=.d)

# Include the dependency files – if these do not exists, make will ignore them
-include $(DEPFILES)

Adding prefixes

SOURCES = src/foo.c src/bar.c

all:
	@echo $(addprefix subdir/,$(SOURCES))       # Use 'addprefix' function  - clearer
	@echo $(foreach V,$(SOURCES),subdir/$(V))   # Using 'foreach'           - powerful but less clear
	@echo $(SOURCES:%=subdir/%)                 # Using substitution        - even more clear when...
	@echo $(SOURCES:%.c=obj/%.o)                #                             ... changing ext as well

Conditionals

There are several types of conditionals in make:

Functions for conditionals
These are like regular if-then instructions, and can be used along with Automatic Variables (i.e. $@ $?...).
main.zip:: nfc.zip $(JAVA_SRC) Makefile
	$(if $(filter %.java,$?),echo 'Indenting (savagely) your code...'; uncrustify --no-backup -c indent.uncrustify.prefs $(filter %.java,$?))
Conditional Parts of Makefiles
These are like conditional directives in C preprocessor. From the documentation:

make evaluates conditionals when it reads a makefile. Consequently, you cannot use automatic variables in the tests of conditionals because they are not defined until recipes are run (see Automatic Variables).

#Conditianals in rule recipes:
libs_for_gcc = -lgnu
normal_libs =

foo: $(objects)
ifeq ($(CC),gcc)
	$(CC) -o foo $(objects) $(libs_for_gcc)
else
	$(CC) -o foo $(objects) $(normal_libs)
endif

#Conditionals in variable definitions:
ifeq ($(CC),gcc)
	libs=$(libs_for_gcc)
else
	libs=$(normal_libs)
endif
     
foo: $(objects)
	$(CC) -o foo $(objects) $(libs)

Echo while 1st pass expansion

To have make echo some information while in the 1st pass expansion phase (i.e. before running target recipe), one can use a dummy variable and $(shell ...) function:

x := $(shell echo >&2 'ARM Based Posix system with J9DROP libraries')

However a standard echo works as well:

$(shell echo >&2 'ARM Based Posix system with J9DROP libraries')

Note that >&2 is necessary otherwise you get a strange error missing separator from make.

Alternatively one can use info:

$(info This is an information message.)

Change directory permanently

Reference: [2]

CHDIR_SHELL := $(SHELL)
$(info CHDIR_SHELL is $(CHDIR_SHELL))
define chdir
   $(eval _D=$(firstword $(1) $(@D)))
   $(info $(MAKE): cd $(_D)) $(eval SHELL = cd $(_D); $(CHDIR_SHELL))
endef

# Can change permanently directory during the 1st pass...
$(eval cd some_dir2; /bin/sh)
$(shell echo >&2 NOW PWD is $$PWD)

all:
# Or in a rule recipe...
	$(call chdir,some_dir)
	echo "I'm now always in some_dir"
	echo gcc -Wall -o myTest myTest.c
	echo $$PWD

some_dir/myTest:
	$(call chdir)
	echo "I'm now always in some_dir"
	echo gcc -Wall -o myTest myTest.c
	echo $$PWD

Using Function Variables

Function variables can either be defined by assigned or using define... endef construct.

Assignments must be deferred (i.e. = and not :-), so that function is not evaluated at the time it is defined but only when used. Also if it contains comment symbol (#), these must be escaped:

makefilecfg = $(shell echo "Creating configuration file '$R$(MAKEFILE_CFG)$Z'." >&2; \
	mkdir -p $(BAD) >&2; \
	echo "\# Common Platform custom configuration file." >$(MAKEFILE_CFG) ;\
	echo "\# CP build system includes this file to know which components/variants to build." >>$(MAKEFILE_CFG); \
	echo "\# See file 'tools/checkcfg.mk' for list of available components/variants." >>$(MAKEFILE_CFG); \
	echo "CPU :=$(CPU)"  >>$(MAKEFILE_CFG); \
	echo "OSAL:=$(OSAL)" >>$(MAKEFILE_CFG); \
	echo "JAVA:=$(JAVA)" >>$(MAKEFILE_CFG); \
	echo "NFC :=$(NFC)"  >>$(MAKEFILE_CFG); \
	echo "" >&2; \
	echo "You can now run '$Gmake$Z' to build the project with this configuration," >&2; \
	echo "or first edit the configuration file to customize this build." >&2 )

all:
	@(call $(makefilecfg))

The define... endef is a bit simpler:

define makefilecfg
	$(shell echo "Creating configuration file '$R$(MAKEFILE_CFG)$Z'." >&2;
	mkdir -p $(BAD) >&2;
	echo "# Common Platform custom configuration file." >$(MAKEFILE_CFG);
	echo "# CP build system includes this file to know which components/variants to build." >>$(MAKEFILE_CFG);
	echo "# See file 'tools/checkcfg.mk' for list of available components/variants." >>$(MAKEFILE_CFG);
	echo "CPU :=$(CPU)"  >>$(MAKEFILE_CFG);
	echo "OSAL:=$(OSAL)" >>$(MAKEFILE_CFG);
	echo "JAVA:=$(JAVA)" >>$(MAKEFILE_CFG);
	echo "NFC :=$(NFC)"  >>$(MAKEFILE_CFG) )
endef

all:
	@(call $(makefilecfg))

Use $(info ...) instead of $(shell echo ...)

Use $(info ...) instead of $(shell echo ...) to print information message. The later invokes a new shell process for each printed message, and is notably slower on Cygwin platform for instance.

Test whether a file exists

ifeq "$(wildcard $(FILENAME))" ""
  $(error File $(FILENAME) does not exist.)
endif

Substitute space character into comma

Due to make syntax rules, converting a space-separated list into a comma-separated one is not straightforward. From GNU Make manual (§8.1):

Commas and unmatched parentheses or braces cannot appear in the text of an argument as written; leading spaces cannot appear in the text of the first argument as written. These characters can be put into the argument value by variable substitution. First define variables comma and space whose values are isolated comma and space characters, then substitute these variables where such characters are wanted [...]

The trick given in the manual is as follows:

comma:= ,
empty:=
space:= $(empty) $(empty)
foo:= a b c
bar:= $(subst $(space),$(comma),$(foo))
# bar is now 'a,b,c'.
# Note: slightly better is $(subst $(space),$(comma),$(strip $(foo)))

Escaping space, comma and other characters

As seen in the previous tip, one need to use a trick to pass spaces or commas as function parameters. Make however is very liberal regarding variable name, and one can even use a variable name that consists in a space character [3]:

# Defining the $  or $( ) variable which has the value =
empty :=
space := $(empty) $(empty)
$(space) := $(space)

# Defining the $, or $(,) variable which has the value =
comma := ,
$(comma) := ,

# Defining the $= or $(=) variable which has the value =
equals := =
$(equals) := =

# Define the $# or $(#) variable which has the value #
hash := \#
$(hash) := \#

# Define the $: or $(:) variable which has the value :
colon := :
$(colon) := :

# Define the $($$) variable which has the value $
dollar := $$
$(dollar) := $$

; := ;
% := %

For instance:

$(info [$( )] [$ ] [$(,)] [$,] [$(=)] [$=])
bar:= $(subst $ ,$,,$(foo))

Regarding space, some developers say at [4] that the following trick does not work:

# The following might not work (space would actually contain a newline?)
space := 
space +=

Define multi-line strings

The best method is to use the define ... endef construct and export the variable, and then reference it as a shell variable with $$VARIABLE (see [5]):

define USAGE_TEXT
Usage: ...

Option:
  First    this and that
  Second   Can refer to make $(variable) too
endef

export USAGE_TEXT
usage:
	@echo >&2 "$$USAGE_TEXT"

Using $(USAGE_TEXT) would not work, or require escaping the end of line, add '\n', and use echo -e.

Alternatively, for warning messages, we can do $(warning $(USAGE_TEXT)).

Passing target to make sub-module

The same technique is illustrated in Make Manual at §3.6. We provide a default recipe to unknown target through a catch-all pattern rule, hence catching in $@ the name of the target given on the command-line.

# file 'Makefile'

first:
	@echo target $@

# There is always an implicit 'make Makefile' issued by make.
# We add this target so that it is not caught by generic rule below
Makefile:
	@echo special target $@

# Try 'make second; touch second; make second'
# Thanks to force, target 'second' is always done
%: force
	@echo target $@
	@$(MAKE) $@ -f Makefile.mk               # pass $@ as target to Makefile.mk

# Don't forget semi-colon
force: ;

third:
	@echo target $@

Color messages

# We first fefine ESC - \033 doesn't work in $(info ...). The 2nd one is a verbatim escape char. The last one is most portable (dash and bash)
# ... then we define the color code. DO NOT ADD TRAILING COMMENTS OR THIS WILL ADD TRAILING SPACES.
# ESC := \033
# ESC := ^[
# ESC := $(shell echo "\033")
ESC := $(shell printf '\e')
R := $(ESC)[0;31m
G := $(ESC)[0;32m
B := $(ESC)[0;34m
Z := $(ESC)[0;0m

$(info A $Rcolorful $Zand $Gbeautiful $Bmessage$Z!)
all:
    echo "$Rcolorful $Zand $Gbeautiful $Bmessage$Z!"

Note that using \033 only works in shell echo command. The other two commands works in both $(info ...) and shell echo command.

Color messages only if output to a tty

See my answer on StackOverflow

(new) Use MAKE_TERMOUT / MAKE_TERMERR variables (make 4.1+)

This is the best method if using make 4.1+.

# Simple solution
ifdef MAKE_TERMOUT
ifdef MAKE_TERMERR
# DO NOT ADD TRAILING COMMENTS OR THIS WILL FAIL BECAUSE OF TRAILING SPACES
ESC := $(shell printf '\e')
R := $(ESC)[31m
Z := $(ESC)[0;0m
endif
endif

$(info [info  ] $Rred$Z black)
$(shell echo >&2 "[shell ] $Rred$Z black")
all:
	@echo "[recipe] $Rred$Z black"
	MAKE_TERMERR= $(MAKE) 2> /dev/null
	MAKE_TERMOUT= $(MAKE) | cat

piped:
	@echo "[recipe] $Rred$Z black"
Detect TTY in a custom make script

We simply write a simple shell script that intercepts the call to the native make, detects the TTY, and defines a variable appropriately. The main advantages of this solution:

  • simplicity,
  • works for echo, $(info ...) and alike, $(shell ...) commands,
  • no modification required in the recipes,
  • no extra process spawned at each echo, which on some platform can be quite slow (eg. Cygwin).
ifdef IS_TTY
# DO NOT ADD TRAILING COMMENTS OR THIS WILL FAIL BECAUSE OF TRAILING SPACES
ESC := $(shell printf '\e')
R := $(ESC)[31m
Z := $(ESC)[0;0m
endif

$(info [info  ] $Rred$Z black)
$(shell echo >&2 "[shell ] $Rred$Z black")
all:
	@echo "[recipe] $Rred$Z black"

Example of custom make script to add in path (eg. /usr/local/bin/make):

#! /bin/bash

[ -t 1 ] && IS_TTY=1 || IS_TTY=0
exec /usr/bin/make IS_TTY=$IS_TTY "$@"

This method provides the expected output in all these cases:

make                  # Colored
make >&2              # Colored
make | cat            # NOT colored
make >tmp && cat tmp  # NOT colored
Detect in recipe and call the Makefile recursively

In some cases, it might not be possible to intercept the call to make or replace it with a custom script. In this solution, we fix that limitation by detecting the TTY in a first pass, and then calling the same Makefile recursively with the appropriate variable set.

ifndef IS_TTY
.SILENT:
%:
    @[ -t 1 ] && IS_TTY=1 || IS_TTY=0; $(MAKE) IS_TTY=$$IS_TTY "$@"
xyz:
    @[ -t 1 ] && IS_TTY=1 || IS_TTY=0; $(MAKE) IS_TTY=$$IS_TTY
else

ifeq ($(IS_TTY),1)
# DO NOT ADD TRAILING COMMENTS OR THIS WILL FAIL BECAUSE OF TRAILING SPACES
ESC := $(shell printf '\e')
R := $(ESC)[31m
Z := $(ESC)[0;0m
endif

$(info [info  ] $Rred$Z black)
$(shell echo >&2 "[shell ] $Rred$Z black")
recursive:
    @echo "[recipe] $Rred$Z black"

endif
Strip the escape sequences at each echo

We test at each echo command whether the output is a TTY, and we strip the escape sequences if not so (using https://superuser.com/questions/380772/removing-ansi-color-codes-from-text-stream). To reduce the spam on the line, we use a make variable $(STRIPESC). The solution is quite simple, but it only works for echo command, and spawns an extra process at each echo. It also requires to edit every recipe with an echo.

# DO NOT ADD TRAILING COMMENTS OR THIS WILL FAIL BECAUSE OF TRAILING SPACES
ESC := $(shell printf '\e')
R := $(ESC)[31m
Z := $(ESC)[0;0m
STRIPESC:=( [ -t 1 ] && cat || sed 's/\x1b\[[0-9;]*m//g' )

# $(info ...) not supported
# $(shell echo >&2 ...) not supported
if_tty:
  @echo "[recipe] $Rred$Z black" | $(STRIPESC)
Use an external echo command to strip escape sequences

This is similar to the solution above except that the syntax is a bit more lightweight. The pros/cons are the same. Also, in the example below, I generate the external echo in the makefile itself. In a standard build, you would provide this command as an external tool in some standard path.

# DO NOT ADD TRAILING COMMENTS OR THIS WILL FAIL BECAUSE OF TRAILING SPACES
ESC := $(shell printf '\e')
R := $(ESC)[31m
Z := $(ESC)[0;0m

$(shell echo "#! /bin/bash" > echotty)
$(shell echo '[ -t 1 ] && echo "$$@" || echo "$$@" | sed "s/\x1b\[[0-9;]*m//g"' >> echotty)
$(shell chmod a+x echotty)

# $(info ...)
# $(shell echo >&2 ...)
.PHONY: echotty
echotty:
  @./echotty "[recipe] $Rred$Z black"

Detect if Make runs in an interactive shell

This one is simpler, and simply consists in testing whether FD 0 is a TTY [6]:

INTERACTIVE:=$(shell [ -t 0 ] && echo 1)

ifdef INTERACTIVE
# is a terminal
else
# cron job
endif

Silent make by default, unless verbose requested

Make by default has echoing, i.e. it prints recipe command before executing them. This can be silented by

  • Prefixing command with @
  • Using option --silent or -s
  • Using special target .SILENT

A much better way is to disable echoing by default, and print relevant commands only in some case [7]:

#Silent unless verbose (also for sub-make calls)
ifndef VERBOSE
.SILENT:
endif

# Same but syntax trick
$(VERBOSE).SILENT:

# Same but using MAKEFLAGS
ifndef VERBOSE
MAKEFLAGS += --silent
endif

Doing a make VERBOSE=1 will print the recipe commands (unless if they are prefixed with @). All the tricks above also affect sub-make calls.

Subst a variable only if defined

Note that by default pattern substitution only occurs if the given variable is not empty:

$(shell touch foo.ld foo.c bar.c)

FOO=foo.ld

all: foo bar

foo: foo.c foo.c $(FOO)
    @echo gcc $(FOO:%=-T %) $(filter %.c,$^) -o $@
    @echo gcc $(patsubst %,-T %,$(filter %.ld,$^)) $(filter %.c,$^) -o $@

bar: bar.c $(BAR)
    @echo gcc $(BAR:%=-T %) $(filter %.c,$^) -o $@
    @echo gcc $(patsubst %,-T %,$(filter %.ld,$^)) $(filter %.c,$^) -o $@

This produces:

gcc -T foo.ld foo.c bar.c -o foo
gcc -T foo.ld foo.c bar.c -o foo
gcc bar.c -o bar
gcc bar.c -o bar

Enable or disable parallel build with MAKEFLAGS

Add flags -j to MAKEFLAGS. When there are many targets, it is better to limit a bit the number of parallel jobs (eg. -j64).

Be careful that not all Makefile are designed to run in parallel. Ideally, to prevent parallel execution, the Makefile must contain the .NOTPARALLEL target:

# This will prevent parallel execution of this Makefile
.NOTPARALLEL:

To preserve output consistency, we can also use -O to buffer output. For long build, -O often means there will be no output for a long time, so a better compromise is -Oline, which at least preserve output consistency for most parsers (like IDE that parses make output for errors).

export MAKEFLAGS="-j64 -Oline"            # 64 jobs, preserve line consistency
make

Alternatively we can add -Onone to MAKEFLAGS

# Make sure output-sync (-Oline) is disabled
MAKEFLAGS += -Onone

To disable parallel build for one target only:

# We don't want buffer when debugging
.PHONY: gdb gdb-nobuf
gdb: MAKEFLAGS+=-Onone
gdb: $(APP)
	$(MAKE) gdb-nobuf

gdb-nobuf:
	gdb $(APP)

Another method:

auto-my_target:
	[ "$$MAKEFLAGS" != "-j1" ] && $(MAKE) $@ MAKEFLAGS=-j1 || $(@:auto-%=%)/gdb - -x gdbinit_auto

Use ccache to cache build objects

See ccache.

Use target specific variable

One can define custom variables that are defined only for specific targets. The recipe syntax requires that variables be defined independently of other dependencies.

target: SOME_VAR = SOME_VALUE
target: src_a.c src_b.c
    $(CC) $^ -o $@

Note that variables are defined on a separate line. To have the variable exported in the environment, use export keyword:

target: export SOME_VAR = SOME_VALUE
target: src_a.c src_b.c
    $(CC) $^ -o $@

Disable implicit rules

Implicit rules are those that are defined by default in Make (like .c to .o conversion.

To disable them, just redefine .SUFFIXES variable:

.SUFFIXES:

Detect if running on Linux or Windows

We can use OS variable on Windows, but we cannot use OSTYPE on Linux (not defined in Makefile). So from Stackoverflow, the source of all wisdom:

ifeq ($(OS),Windows_NT)
    CCFLAGS += -D WIN32
    ifeq ($(PROCESSOR_ARCHITEW6432),AMD64)
        CCFLAGS += -D AMD64
    else
        ifeq ($(PROCESSOR_ARCHITECTURE),AMD64)
            CCFLAGS += -D AMD64
        endif
        ifeq ($(PROCESSOR_ARCHITECTURE),x86)
            CCFLAGS += -D IA32
        endif
    endif
else
    UNAME_S := $(shell uname -s)
    ifeq ($(UNAME_S),Linux)
        CCFLAGS += -D LINUX
    endif
    ifeq ($(UNAME_S),Darwin)
        CCFLAGS += -D OSX
    endif
    UNAME_P := $(shell uname -p)
    ifeq ($(UNAME_P),x86_64)
        CCFLAGS += -D AMD64
    endif
    ifneq ($(filter %86,$(UNAME_P)),)
        CCFLAGS += -D IA32
    endif
    ifneq ($(filter arm%,$(UNAME_P)),)
        CCFLAGS += -D ARM
    endif
endif

Print newlines reliably in sh/bash/dash

Use printf instead of echo to have more portable results [8]:

all:
    printf "hello\nworld\n"                # Works for any SHELL

Use wildcard with file name with spaces

$(wildcard ...) works with filenames with space if they are properly escaped:

empty:=
space:= $(empty) $(empty)
ifeq "$(wildcard $(subst $(space),\$(space),$(SOME_SPACED_PATH))/Include)" ""
$(info $(shell echo -e '$b$RERROR: PU path not found ($(SOME_SPACED_PATH)).$Z'))
$(error ERROR: path not found.)
endif

Strip trailing slash from directory

The simplest (also produce . if file has no directory part):

FILE:=foo/bar
PARENT:=$(patsubst %/,%,$(dir $(F)))

Force digest computation on up-to-date targets

We can use the double-colon rule to tell make to always redo some recipes, without triggering a complete rebuild (unlike .PHONY targets):

AXF:=$(DIR)/foo.axf
HEX:=$(DIR)/foo.hex
$(HEX):: $(AXF)
	$(OBJCOPY) $(OBJCOPY_FLAGS) $@ $<

ifdef DIGEST
# According to manual, this double-colon rule w/o prerequisite is always executed, and always after the one on top...
$(HEX)::
	sha1sum $@
endif

Frequent Mistakes

Use TABS to prefix commands in rules, *NOT* SPACES
Each rule in a makefile gives the set of commands that must be executed to build a given target from a given set of dependencies. These commands MUST be indented with a TAB character, not with SPACES. So make sure that the editor does not automatically change these tabs into spaces.
Don't call sub-project Makefile, include it
Calling subproject makefiles, so-called recursive make, has a huge performance penalty however the build (see Recursive Make Considered Harmful)
Don't add space around = (equal sign) in variable substitution
Do $(SOURCES:%.c=%.o), not $(SOURCES:%.c = %.o)
Use COMMA to separate parameters in $(call), not SPACE
Do $(call FCT,$(param)), not $(call FCT $(param)). The later will fail silently!
Use $(shell echo ...) outside of a recipe
Using $(shell echo ...) in the body of the Makefile will generate an error Missing separator, because the output of the shell command is injected as is in the Makefile. Instead, either redirect with $(shell echo &2 ...), or use $(info $(shell echo ...)), but oftentimes $(shell echo ) is superfluous. [9]
MUST add space after ifeq, as in ifeq ($OS,Windows_NT)
Otherwise make will complain with *** missing separator. Stop.

Bugs

ABS_DIRS := $(filter /%,$(DIRS)) $(abspath $(filter-out /%,$(DIRS)))