automake: how to portably throw an error and aborting the target

Question

Depending on a condition checked for in the configure script, I want to throw an error, thus aborting the target. (i.e. refuse to compile tests on 'make check', if the test framework is not installed)

The behavior I want to achieve is, that when you call make check, it will directly print diagnostics and exits.

However, if the Makefile contains:

# if check is missing, don't do anything at all
if MISSING_CHECK
check:
    @echo "tests are disabled due to missing check"
    $(error tests are disabled due to missing check)

endif

Automake answers this:

automake-1.16: warnings are treated as errors
Makefile.am:9: warning: error tests are disabled due to missing check: non-POSIX variable name
Makefile.am:9: (probably a GNU make extension)
make: *** [Makefile:347: ../Makefile.in] Error 1

So the question is, how I could portably can throw an error.

---

Thanks to @John Bollinger /Makefile.am now looks like this:

## Process this file with automake to produce Makefile.in

SUBDIRS = . src tests

# if check is missing, don't do anything at all
if MISSING_CHECK
check:
    @echo "tests are disabled due to missing check"
    @exit 1

endif

ACLOCAL_AMFLAGS = --install -I build-macro

And in /configure.ac there is:

AM_CONDITIONAL([MISSING_CHECK], [test x${enable_tests} = xno])

where enable_tests is either set to yes or no.

/tests/Makefile.am contains this:

# Process this file with automake to produce Makefile.in

AM_CFLAGS = -Wall -Wextra $(CHECK_CFLAGS)

TESTS =
TESTS += test_integer

check_PROGRAMS = $(TESTS)
[...]

If I execute make check, it currently looks like this:

Making check in .
make[1]: Entering directory '/tmp/build'
make[1]: Nothing to be done for 'check-am'.
make[1]: Leaving directory '/tmp/build'
Making check in src
make[1]: Entering directory '/tmp/build/src'
Making check in lib
make[2]: Entering directory '/tmp/build/src/lib'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/tmp/build/src/lib'
make[2]: Entering directory '/tmp/build/src'
make[2]: Leaving directory '/tmp/build/src'
make[1]: Leaving directory '/tmp/build/src'
Making check in tests
make[1]: Entering directory '/tmp/build/tests'
make  
make[2]: Entering directory '/tmp/build/tests'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/tmp/build/tests'
make  check-TESTS
make[2]: Entering directory '/tmp/build/tests'
make[3]: Entering directory '/tmp/build/tests'
============================================================================
Testsuite summary for libvector 0.9
============================================================================
# TOTAL: 0
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[3]: Leaving directory '/tmp/build/tests'
make[2]: Leaving directory '/tmp/build/tests'
make[1]: Leaving directory '/tmp/build/tests'
tests are disabled due to missing check
make: *** [Makefile:820: check] Error 1

Which seams to tell a user, that all tests have succeed. But I want the error to be thrown before the Testsuite is tried to run.

Chosen solution

As it turned out, that overwriting the test-harness is quite difficult, because it is unconditionally added before any custom rules and to do so you have to depend on implementation defined characteristics of automake. See John Bollinger's answer for more details.

That's why instead of overwriting the test harness, I add only a single shell script to it, in case the tests aren't to be compiled.
/tests/Makefile.am:

# if check is missing, run script to inform the user of this
if MISSING_CHECK
check_SCRIPTS = no_test.sh
# allow the script to echo to stderr
AM_TESTS_FD_REDIRECT = 9>&2
endif

if !MISSING_CHECK
check_PROGRAMS =
check_PROGRAMS += test_integer

endif

TESTS = $(check_SCRIPTS) $(check_PROGRAMS)

The script is generated by:
/configure.ac:

AC_CONFIG_FILES([tests/no_test.sh], [chmod +x tests/no_test.sh])

from:
/tests/no_test.sh.in:

#!/bin/bash

# Inform the user that tests are disabled and how to change that.
echo "No tests created due to missing check or user preference." >&2;
echo "No tests created due to missing check or user preference." >&9;
echo "To enable tests, run configure with --enable-tests=yes" >&2;
echo "To enable tests, run configure with --enable-tests=yes" >&9;
exit 1

Note that the user notification is issued twice as the stderr/stdout of the script is redirected to logfiles by the parallel test harness. &9 is then redirected to the real stderr by AM_TESTS_FD_REDIRECT = 9>&2. For the serial test harness it is simply directedto the real stderr and AM_TESTS_FD_REDIRECT is just ignored. See: https://www.gnu.org/software/automake/manual/html_node/Testsuite-Environment-Overrides.html

The toplevel /Makefile.am is not changed at all.

Answer 1

The Autotools have a strong focus on portability across UNIX-like systems. You can fight that in various ways, but then the question arises of why you are using the Autotools in the first place.

However, Automake has better ability to analyze your Makefile.am with respect to portability than Autoconf has to analyze your configure.ac in the same light. If you write custom rules into your Makefile.am (or into another file that it includes) then you should stick to portable features, which mostly means those covered by the POSIX specifications for make. GNU extensions are largely out, especially all the make "functions" it provides, such as $(error).

how I could portably can throw an error.

Generally speaking, in make recipes you should focus on shell features. In this case, unless you employ the make syntax to specify otherwise, a recipe will fail, causing make to stop and report an error, if any of the commands in it fails. There is a standard command whose entire purpose is to fail: false. Alternatively, exit 1 will terminate the shell with a failing exit status, which would serve the same purpose in this case. Thus, you might do something like this:

# if check is missing, don't do anything at all
if MISSING_REQUIREMENTS
foo:
        @echo "optional component foo cannot be built due to missing requirements"
        @exit 1

endif

Alternatively, you could consider whether it would be sufficient to just print the diagnostic message, without failing.

---

HOWEVER, what you actually want to do is alter the behavior of a target (check) that belongs to Automake. This is allowed, but somewhat fraught. In particular, the documentation remarks that

Overriding Automake rules is often inadvisable, particularly in the topmost directory of a package with subdirectories.

You have exercised exactly the least-advisable case for overriding Automake rules, and "inadvisable" here manifests as "it won't have the result that you probably expect".

If you dig into the Makefile generated by configure, you will discover that the behavior you observe results from the fact that the top-level check target has a prerequisite (check-recursive) that implements the recursion. The prerequisite is built before the override recipe for target check is executed.

In a recursive Automake makefile, the local, non-recursive version of each standard target is named with an -am suffix. You can in fact see in your output that check-am in the top directory is the first target built when you make check.

Armed with that knowledge, you could obtain the behavior you are after by overriding check-am instead of check:

SUBDIRS = . src tests

# if check is missing, don't do anything at all
if MISSING_CHECK
check-am:
    @echo "tests are disabled due to missing check"
    @exit 1

endif

But note that

1. To get the result you want this way, you have to process the top-level directory first, which often conflicts with what you want for other purposes.

2. We are now depending on undocumented Automake implementation details.

3. Generally speaking, it is supported to run make in any of the subdirectories, and doing so will bypass any overrides or error generation implemented at the top level.

Overall, my recommendation is don't do that.

DO have configure emit a warning if libcheck is not available (AM_COND_IF can help with that). This is the best and most timely way to notify a builder of the issue.

DO suppress definition of any test cases when CHECK_MISSING is true, so that you don't hit builders with weird build failures in the event that they try a make check despite the warning from configure.

Possibly do even override the top-level check target to emit an explanatory message about the absence of test cases, maybe something more prominent, such as:

if MISSING_CHECK
check:
    @echo ============================================================================
    @echo "NOTICE: no test cases are available because libcheck is not installed"
    @echo ============================================================================

endif

And if you feel you must, do cause make check to fail in this case, by the means already described.

But DO NOT sweat the fact that an empty test suite is executed and summarized if a builder executes a make check despite having already been notified that no tests are available.