[tests] failures against Pytest 7.4.0
Recently released Pytest 7.4.0 broke several tests in Bind's tests suite. For example,
D:statschannel:============================= test session starts ==============================
D:statschannel:platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0 -- /usr/bin/python3
D:statschannel:cachedir: .pytest_cache
D:statschannel:hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/__w/1/s/rpmbuild/BUILD/bind-9.16.42/bin/tests/system/statschannel/.hypothesis/examples')
D:statschannel:rootdir: /__w/1/s/rpmbuild/BUILD/bind-9.16.42/bin/tests/system/statschannel
D:statschannel:plugins: hypothesis-6.75.3
D:statschannel:collecting ... collected 0 items / 1 error
D:statschannel:
D:statschannel:==================================== ERRORS ====================================
D:statschannel:________________________ ERROR collecting tests_json.py ________________________
D:statschannel:ImportError while importing test module '/__w/1/s/rpmbuild/BUILD/bind-9.16.42/bin/tests/system/statschannel/tests_json.py'.
D:statschannel:Hint: make sure your test modules/packages have valid Python names.
D:statschannel:Traceback:
D:statschannel:/usr/lib64/python3.11/importlib/__init__.py:126: in import_module
D:statschannel: return _bootstrap._gcd_import(name[level:], package, level)
D:statschannel:tests_json.py:19: in <module>
D:statschannel: import pytest_custom_markers
D:statschannel:E ModuleNotFoundError: No module named 'pytest_custom_markers'
D:statschannel:!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
D:statschannel:=============================== 1 error in 0.21s ===============================
or
D:wildcard:==================================== ERRORS ====================================
D:wildcard:_______________ ERROR at setup of test_wildcard_rdtype_mismatch ________________
D:wildcard:file /__w/1/s/rpmbuild/BUILD/bind-9.16.42/bin/tests/system/wildcard/tests_wildcard.py, line 78
D:wildcard: @given(
D:wildcard: label=binary(min_size=1, max_size=63),
D:wildcard: rdtype=integers(min_value=0, max_value=65535).filter(is_nonexpanding_rdtype),
D:wildcard: )
D:wildcard: def test_wildcard_rdtype_mismatch(label, rdtype, named_port):
D:wildcard:E fixture 'named_port' not found
D:wildcard:> available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
D:wildcard:> use 'pytest --fixtures [testpath]' for help on them.
D:wildcard:
D:wildcard:/__w/1/s/rpmbuild/BUILD/bind-9.16.42/bin/tests/system/wildcard/tests_wildcard.py:78
D:wildcard:____________________ ERROR at setup of test_wildcard_match _____________________
D:wildcard:file /__w/1/s/rpmbuild/BUILD/bind-9.16.42/bin/tests/system/wildcard/tests_wildcard.py, line 93
D:wildcard: @given(label=binary(min_size=1, max_size=63))
D:wildcard: def test_wildcard_match(label, named_port):
D:wildcard:E fixture 'named_port' not found
D:wildcard:> available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
D:wildcard:> use 'pytest --fixtures [testpath]' for help on them.
https://docs.pytest.org/en/stable/changelog.html#pytest-7-4-0-2023-06-23
#11043: When --confcutdir is not specified, and there is no config
file present, the conftest cutoff directory (--confcutdir) is now set to
the rootdir. Previously in such cases, conftest.py files would be probed
all the way to the root directory of the filesystem. If you are badly
affected by this change, consider adding an empty config file to your
desired cutoff directory, or explicitly set --confcutdir.
Possible fix for bind 9.16 branch looks like:
--- a/bin/tests/system/run.sh
+++ b/bin/tests/system/run.sh
@@ -237,7 +237,7 @@ if [ $status -eq 0 ]; then
if start_servers; then
run=$((run+1))
test_status=0
- (cd "$systest" && "$PYTEST" -rsxX -v "$test" "$@" || echo "$?" > "$test.status") | SYSTESTDIR="$systest" cat_d
+ (cd "$systest" && "$PYTEST" --confcutdir "$TOP/bin/tests/system" -rsxX -v "$test" "$@" || echo "$?" > "$test.status") | SYSTESTDIR="$systest" cat_d
if [ -f "$systest/$test.status" ]; then
if [ "$(cat "$systest/$test.status")" != "5" ]; then
test_status=$(cat "$systest/$test.status")
This patch was tested and helped me.