Resolve "make gtests, shell tests, python tests run sequentially (not concurrently)"
Closes #1642 (closed)
This change is mostly to not force make distcheck -j 1
and to allow make distcheck -j 2
for faster Jenkins testing.
Managed to do it without splitting into separate directories. All that needs to be done is to add the scripts to TESTS
and to check_SCRIPTS
. Both were missing.
https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html
If the special variable TESTS is defined, its value is taken to be a list of programs or scripts to run in order to do the testing.
https://www.gnu.org/software/automake/manual/html_node/Scripts.html
Scripts that need not be installed can be listed in noinst_SCRIPTS, and among them, those which are needed only by ‘make check’ should go in check_SCRIPTS.
Apart from that, I felt like these Makefile.am
s could use more ordering. Here is the logic that I used:
- some have
SUBDIRS = .
, some don't. For consistency, I added it everywhere. It shouldn't make a real diference. - ordered automake statements in the order that you would normally call upon them as an autotools user:
- EXTRA_DIST -
make distdir
- TESTS_ENVIRONMENT -
make check
(needed when called with valgrind I think) - SHTESTS + check_SCRIPTS + TESTS -
make check
- DISTCLEANFILES -
make clean
- noinst_SCRIPTS -
make install
- and then for gtest (C++ tests):
- PROGRAMTESTS -
make check
with gtest enabled - DISTCLEANFILES -
make clean
with gtest enabled - noinst_SCRIPTS -
make install
with gtest enabled
- EXTRA_DIST -
The shell and python tests are now considered proper automake tests instead of some random strings that we run alongside make check
. You'll see in the libraries that contain shell tests or python tests that instead of 1 test passed
you get All 2 tests passed
refering to C++ tests (1) and shell/python tests (2). Not sure on what implications this will have, but I'm thinking it can only be good.
Apart from that, some tests now failed because they some variables different paths in make distcheck
versus make check
. In one instance, these variables pointed to the installation directory which was empty, and is most of the time so. So to avoid this error, I replaced a conditional assignment of KEA_*_DIR to always point to builddir. This seems to have solved the issue.
distcheck run in jenkins on this branch: https://jenkins.aws.isc.org/view/Kea-manual/job/kea-manual/job/distcheck/22/