Out-of-tree builds fail running make test
Summary
Out-of-tree builds fail running system tests
BIND version affected
Recent main branch, commit 7838ca3f.
Steps to reproduce
- Have Fedora 41 Rawhide (or Fedora 40)
git clone https://gitlab.isc.org/isc-projects/bind9.git
- cd bind9
git checkout main
autoreconf -fsi
mkdir build
cd build
../configure --enable-developer
make -j4
sudo bin/tests/system/ifconfig.sh up
cp -auv ../bin/tests/system/* bin/tests/system
LANG=C.UTF-8 make test TESTS=digdelv V=1
What is the current bug behavior?
First, it fails to correctly run convert-junit-to-trs.py:
make[8]: Entering directory '/root/bind9/build/bin/tests/system'
../../../../bin/tests/system/custom-test-driver: line 149: ./convert-junit-to-trs.py: No such file or directory
When I fixed that with:
diff --git a/bin/tests/system/custom-test-driver b/bin/tests/system/custom-test-driver
index 278cb377e2b..7f2e23be292 100755
--- a/bin/tests/system/custom-test-driver
+++ b/bin/tests/system/custom-test-driver
@@ -58,6 +58,8 @@ junit_file= # Where to save pytest junit output.
expect_failure=no
color_tests=no
enable_hard_errors=yes
+srcdir="$(dirname -- $0)"
+convert_junit="$srcdir/convert-junit-to-trs.py"
verbose=no
while test $# -gt 0; do
case $1 in
@@ -146,7 +148,10 @@ else
fi
# Run junit to trs converter script.
-./convert-junit-to-trs.py $junit_file >$trs_file
+if test $verbose = yes; then
+ echo "# pwd: $(pwd)"
+fi
+$convert_junit $junit_file >$trs_file
estatus=$?
if test $enable_hard_errors = no && test $estatus -eq 99; then
diff --git a/bin/tests/system/run.sh b/bin/tests/system/run.sh
index c6d04855635..fd26ff1e85e 100755
--- a/bin/tests/system/run.sh
+++ b/bin/tests/system/run.sh
@@ -25,6 +25,7 @@ system_test_dir="$1"
shift
(
+ echo "run $system_test_dir from $(pwd); $*"
cd "$system_test_dir" || exit 2
/usr/bin/env python3 -m pytest "$@"
)
It at least tries to execute something, but again, does not work.
make check-TESTS
make[7]: Entering directory '/root/bind9/build/bin/tests/system'
make[8]: Entering directory '/root/bind9/build/bin/tests/system'
run ../../../../bin/tests/system/digdelv from /root/bind9/build/bin/tests/system; --junit-xml /root/bind9/build/bin/tests/system/digdelv.xml
ImportError while loading conftest '/root/bind9/bin/tests/system/conftest.py'.
../conftest.py:26: in <module>
import isctest
../isctest/__init__.py:19: in <module>
from . import vars # pylint: disable=redefined-builtin
../isctest/vars/__init__.py:14: in <module>
from .all import ALL
../isctest/vars/all.py:18: in <module>
from .algorithms import ALG_VARS, CRYPTO_SUPPORTED_VARS
../isctest/vars/algorithms.py:20: in <module>
from .basic import BASIC_VARS
../isctest/vars/basic.py:21: in <module>
"ARPANAME": f"{AC_VARS['TOP_BUILDDIR']}/bin/tools/arpaname",
E KeyError: 'TOP_BUILDDIR'
# pwd: /root/bind9/build/bin/tests/system
usage: convert-junit-to-trs.py [-h] junit_file
convert-junit-to-trs.py: error: argument junit_file: can't open 'digdelv.xml': [Errno 2] No such file or directory: 'digdelv.xml'
FAIL: digdelv
============================================================================
Testsuite summary for BIND 9.21.0-dev
============================================================================
# TOTAL: 0
# PASS: 0
# SKIP: 0
# XFAIL: 0
# FAIL: 0
# XPASS: 0
# ERROR: 0
============================================================================
make[8]: Leaving directory '/root/bind9/build/bin/tests/system'
What is the expected correct behavior?
It either reports out-of-tree tests are not supported or it works and runs tests.
It seems to me automake tests are supposed to be always files. If they were files, not directories, then am__check_pre
in bin/tests/system/Makefile.in
would correctly find dir in build directory and execute test on that. It uses fallback dir=$(srcdir)
, because $tst is not a file, which should not be necessary. Therefore it executes on hidden paths in source dir and works on the first glance.
Relevant configuration files
Relevant logs
# rpm -q gcc automake autoconf make python3-pytest
gcc-14.1.1-7.fc41.x86_64
automake-1.16.5-17.fc41.noarch
autoconf-2.72-3.fc41.noarch
make-4.4.1-8.fc41.x86_64
python3-pytest-8.3.1-1.fc41.noarch
Edited by Petr Menšík