ISC Open Source Projects issueshttps://gitlab.isc.org/groups/isc-projects/-/issues2023-08-29T08:45:39Zhttps://gitlab.isc.org/isc-projects/kea/-/issues/2821array-bounds warnings in ubuntu build2023-08-29T08:45:39ZSeth Arnoldarray-bounds warnings in ubuntu buildHello, I'm reviewing KEA as part of the Ubuntu Main Inclusion request. Part of this is taking a look through the build logs, and there's a few instances of `-Warray-bounds` warnings that concern me:
- MIR process bug: https://bugs.launc...Hello, I'm reviewing KEA as part of the Ubuntu Main Inclusion request. Part of this is taking a look through the build logs, and there's a few instances of `-Warray-bounds` warnings that concern me:
- MIR process bug: https://bugs.launchpad.net/ubuntu/+source/isc-kea/+bug/2002861
- Ubuntu Lunar Build: https://launchpad.net/ubuntu/+source/isc-kea/2.2.0-5ubuntu1/+build/25670367
- Ubuntu Lunar direct build logs: https://launchpadlibrarian.net/655959506/buildlog_ubuntu-lunar-amd64.isc-kea_2.2.0-5ubuntu1_BUILDING.txt.gz
```
...
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib -Wdate-time -D_FORTIFY_SOURCE=2 -DOS_LINUX -I../../.. -I../../.. -Wall -Wextra -Wnon-virtual-dtor -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -pthread -Wno-missing-field-initializers -fPIC -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fdebug-prefix-map=/<<PKGBUILDDIR>>=/usr/src/isc-kea-2.2.0-5ubuntu1 -c libdhcp++.cc -fPIC -DPIC -o .libs/libkea_dhcp___la-libdhcp++.o
In file included from /usr/include/c++/12/string:50,
from /usr/include/boost/asio/ip/address.hpp:19,
from ../../../src/lib/asiolink/io_address.h:15,
from ../../../src/lib/dhcp/duid.h:10,
from ../../../src/lib/dhcp/duid_factory.h:10,
from duid_factory.cc:9:
In function ‘std::__copy_move<true, true, std::random_access_iterator_tag>::__copy_m<unsigned char>(unsigned char const*, unsigned char const*, unsigned char*)unsigned char*’,
inlined from ‘std::__copy_move_a2<true, unsigned char*, unsigned char*>(unsigned char*, unsigned char*, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:495:30,
inlined from ‘std::__copy_move_a1<true, unsigned char*, unsigned char*>(unsigned char*, unsigned char*, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:522:42,
inlined from ‘std::__copy_move_a<true, unsigned char*, unsigned char*>(unsigned char*, unsigned char*, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:529:31,
inlined from ‘std::copy<std::move_iterator<unsigned char*>, unsigned char*>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:620:7,
inlined from ‘std::__uninitialized_copy<true>::__uninit_copy<std::move_iterator<unsigned char*>, unsigned char*>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:147:27,
inlined from ‘std::uninitialized_copy<std::move_iterator<unsigned char*>, unsigned char*>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:185:15,
inlined from ‘std::__uninitialized_copy_a<std::move_iterator<unsigned char*>, unsigned char*, unsigned char>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*, std::allocator<unsigned char>&)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:372:37,
inlined from ‘std::__uninitialized_move_if_noexcept_a<unsigned char*, unsigned char*, std::allocator<unsigned char> >(unsigned char*, unsigned char*, unsigned char*, std::allocator<unsigned char>&)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:397:2,
inlined from ‘std::vector<unsigned char, std::allocator<unsigned char> >::_M_range_insert<__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > > >(__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, std::forward_iterator_tag)void’ at /usr/include/c++/12/bits/vector.tcc:801:9,
inlined from ‘std::vector<unsigned char, std::allocator<unsigned char> >::_M_insert_dispatch<__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > > >(__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, std::__false_type)void’ at /usr/include/c++/12/bits/stl_vector.h:1779:19,
inlined from ‘std::vector<unsigned char, std::allocator<unsigned char> >::insert<__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, void>(__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >)__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >’ at /usr/include/c++/12/bits/stl_vector.h:1481:22,
inlined from ‘isc::dhcp::DUIDFactory::createEN(unsigned int, std::vector<unsigned char, std::allocator<unsigned char> > const&)’ at duid_factory.cc:180:24:
/usr/include/c++/12/bits/stl_algobase.h:431:30: warning: ‘memcpy’ offset 6 is out of the bounds [0, 6] [-Warray-bounds]
431 | __builtin_memmove(__result, __first, sizeof(_Tp) * _Num);
| ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/12/bits/stl_algobase.h:431:30: warning: ‘memcpy’ offset 6 is out of the bounds [0, 6] [-Warray-bounds]
...
```
```
...
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib -Wdate-time -D_FORTIFY_SOURCE=2 -DOS_LINUX -I../../.. -I../../.. -Wall -Wextra -Wnon-virtual-dtor -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -pthread -Wno-missing-field-initializers -fPIC -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fdebug-prefix-map=/<<PKGBUILDDIR>>=/usr/src/isc-kea-2.2.0-5ubuntu1 -c ncr_udp.cc -fPIC -DPIC -o .libs/libkea_dhcp_ddns_la-ncr_udp.o
In file included from /usr/include/c++/12/string:50,
from /usr/include/c++/12/bits/locale_classes.h:40,
from /usr/include/c++/12/bits/ios_base.h:41,
from /usr/include/c++/12/ios:42,
from /usr/include/c++/12/ostream:38,
from /usr/include/c++/12/iostream:39,
from ../../../src/lib/cc/data.h:10,
from ../../../src/lib/dhcp_ddns/ncr_msg.h:15,
from ncr_msg.cc:9:
In function ‘std::__copy_move<true, true, std::random_access_iterator_tag>::__copy_m<unsigned char>(unsigned char const*, unsigned char const*, unsigned char*)unsigned char*’,
inlined from ‘std::__copy_move_a2<true, unsigned char*, unsigned char*>(unsigned char*, unsigned char*, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:495:30,
inlined from ‘std::__copy_move_a1<true, unsigned char*, unsigned char*>(unsigned char*, unsigned char*, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:522:42,
inlined from ‘std::__copy_move_a<true, unsigned char*, unsigned char*>(unsigned char*, unsigned char*, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:529:31,
inlined from ‘std::copy<std::move_iterator<unsigned char*>, unsigned char*>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:620:7,
inlined from ‘std::__uninitialized_copy<true>::__uninit_copy<std::move_iterator<unsigned char*>, unsigned char*>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:147:27,
inlined from ‘std::uninitialized_copy<std::move_iterator<unsigned char*>, unsigned char*>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:185:15,
inlined from ‘std::__uninitialized_copy_a<std::move_iterator<unsigned char*>, unsigned char*, unsigned char>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*, std::allocator<unsigned char>&)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:372:37,
inlined from ‘std::__uninitialized_move_if_noexcept_a<unsigned char*, unsigned char*, std::allocator<unsigned char> >(unsigned char*, unsigned char*, unsigned char*, std::allocator<unsigned char>&)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:397:2,
inlined from ‘std::vector<unsigned char, std::allocator<unsigned char> >::_M_range_insert<__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > > >(__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, std::forward_iterator_tag)void’ at /usr/include/c++/12/bits/vector.tcc:801:9,
inlined from ‘std::vector<unsigned char, std::allocator<unsigned char> >::_M_insert_dispatch<__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > > >(__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, std::__false_type)void’ at /usr/include/c++/12/bits/stl_vector.h:1779:19,
inlined from ‘std::vector<unsigned char, std::allocator<unsigned char> >::insert<__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, void>(__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >)__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >’ at /usr/include/c++/12/bits/stl_vector.h:1481:22,
inlined from ‘isc::dhcp_ddns::D2Dhcid::fromHWAddr(boost::shared_ptr<isc::dhcp::HWAddr> const&, std::vector<unsigned char, std::allocator<unsigned char> > const&)’ at ncr_msg.cc:144:23:
/usr/include/c++/12/bits/stl_algobase.h:431:30: warning: ‘memcpy’ offset 1 is out of the bounds [0, 1] [-Warray-bounds]
431 | __builtin_memmove(__result, __first, sizeof(_Tp) * _Num);
| ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/12/bits/stl_algobase.h:431:30: warning: ‘memcpy’ offset 1 is out of the bounds [0, 1] [-Warray-bounds]
...
```
```
...
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib -I../../../src/bin -I../../../src/bin -Wdate-time -D_FORTIFY_SOURCE=2 -DOS_LINUX -I../../.. -I../../.. -Wall -Wextra -Wnon-virtual-dtor -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -pthread -Wno-missing-field-initializers -fPIC -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fdebug-prefix-map=/<<PKGBUILDDIR>>=/usr/src/isc-kea-2.2.0-5ubuntu1 -c basic_scen.cc -fPIC -DPIC -o .libs/basic_scen.o
In file included from /usr/include/c++/12/memory:63,
from /usr/include/boost/smart_ptr/detail/sp_counted_impl.hpp:35,
from /usr/include/boost/smart_ptr/detail/shared_count.hpp:27,
from /usr/include/boost/smart_ptr/shared_ptr.hpp:17,
from /usr/include/boost/shared_ptr.hpp:17,
from ../../../src/bin/perfdhcp/packet_storage.h:11,
from ../../../src/bin/perfdhcp/test_control.h:10,
from test_control.cc:9:
In function ‘std::__copy_move<true, true, std::random_access_iterator_tag>::__copy_m<unsigned char>(unsigned char const*, unsigned char const*, unsigned char*)unsigned char*’,
inlined from ‘std::__copy_move_a2<true, unsigned char*, unsigned char*>(unsigned char*, unsigned char*, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:495:30,
inlined from ‘std::__copy_move_a1<true, unsigned char*, unsigned char*>(unsigned char*, unsigned char*, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:522:42,
inlined from ‘std::__copy_move_a<true, unsigned char*, unsigned char*>(unsigned char*, unsigned char*, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:529:31,
inlined from ‘std::copy<std::move_iterator<unsigned char*>, unsigned char*>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_algobase.h:620:7,
inlined from ‘std::__uninitialized_copy<true>::__uninit_copy<std::move_iterator<unsigned char*>, unsigned char*>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:147:27,
inlined from ‘std::uninitialized_copy<std::move_iterator<unsigned char*>, unsigned char*>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:185:15,
inlined from ‘std::__uninitialized_copy_a<std::move_iterator<unsigned char*>, unsigned char*, unsigned char>(std::move_iterator<unsigned char*>, std::move_iterator<unsigned char*>, unsigned char*, std::allocator<unsigned char>&)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:372:37,
inlined from ‘std::__uninitialized_move_if_noexcept_a<unsigned char*, unsigned char*, std::allocator<unsigned char> >(unsigned char*, unsigned char*, unsigned char*, std::allocator<unsigned char>&)unsigned char*’ at /usr/include/c++/12/bits/stl_uninitialized.h:397:2,
inlined from ‘std::vector<unsigned char, std::allocator<unsigned char> >::_M_range_insert<__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > > >(__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, std::forward_iterator_tag)void’ at /usr/include/c++/12/bits/vector.tcc:801:9,
inlined from ‘std::vector<unsigned char, std::allocator<unsigned char> >::_M_insert_dispatch<__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > > >(__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, std::__false_type)void’ at /usr/include/c++/12/bits/stl_vector.h:1779:19,
inlined from ‘std::vector<unsigned char, std::allocator<unsigned char> >::insert<__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, void>(__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >)__gnu_cxx::__normal_iterator<unsigned char*, std::vector<unsigned char, std::allocator<unsigned char> > >’ at /usr/include/c++/12/bits/stl_vector.h:1481:22,
inlined from ‘isc::perfdhcp::TestControl::generateClientId(boost::shared_ptr<isc::dhcp::HWAddr> const&) const’ at test_control.cc:408:21:
/usr/include/c++/12/bits/stl_algobase.h:431:30: warning: ‘memcpy’ offset 1 is out of the bounds [0, 1] [-Warray-bounds]
431 | __builtin_memmove(__result, __first, sizeof(_Tp) * _Num);
| ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/12/bits/stl_algobase.h:431:30: warning: ‘memcpy’ offset 1 is out of the bounds [0, 1] [-Warray-bounds]
...
```
I'm nowhere near skilled enough with C++ to determine if this is a false positive from the compiler (or FORTIFY_SOURCE?) and I'm hoping this is a quick and easy one for someone more familiar.
Thankskea2.5.1Razvan BecheriuRazvan Becheriuhttps://gitlab.isc.org/isc-projects/bind9/-/issues/3980Add timeouts to unit tests.2023-05-05T12:17:04ZMark AndrewsAdd timeouts to unit tests.We have had cases where unit tests locks up and we get nothing useful from the CI as it just times out without producing any `artifacts`.
Set an alarm timer and trigger a core dump if the unit test takes too long.We have had cases where unit tests locks up and we get nothing useful from the CI as it just times out without producing any `artifacts`.
Set an alarm timer and trigger a core dump if the unit test takes too long.April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)https://gitlab.isc.org/isc-projects/bind9/-/issues/3978Support using pytest to execute the system tests2023-06-14T10:14:58ZTom KrizekSupport using pytest to execute the system testsAdd an option to execute the entire system test suite with pytest. This should co-exist along with the legacy runner which will be removed at a later point.
For reasons, benefits and timeline, refer to the meta issue [#3810](https://git...Add an option to execute the entire system test suite with pytest. This should co-exist along with the legacy runner which will be removed at a later point.
For reasons, benefits and timeline, refer to the meta issue [#3810](https://gitlab.isc.org/isc-projects/bind9/-/issues/3810).June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Tom KrizekTom Krizekhttps://gitlab.isc.org/isc-projects/bind9/-/issues/3968ThreadSanitizer: data race lib/dns/catz.c:2257:2 in dns__catz_update_cb2023-04-20T10:54:36ZMichal NowakThreadSanitizer: data race lib/dns/catz.c:2257:2 in dns__catz_update_cbJob [#3270428](https://gitlab.isc.org/isc-private/bind9/-/jobs/3270428) failed for [71f6ce0cafdd71cb720701e728c6e2e4f6587233](https://gitlab.isc.org/isc-private/bind9/-/commit/71f6ce0cafdd71cb720701e728c6e2e4f6587233).
This happened on ...Job [#3270428](https://gitlab.isc.org/isc-private/bind9/-/jobs/3270428) failed for [71f6ce0cafdd71cb720701e728c6e2e4f6587233](https://gitlab.isc.org/isc-private/bind9/-/commit/71f6ce0cafdd71cb720701e728c6e2e4f6587233).
This happened on `bind-9.18-sub`, so while I don't think it's the case, we just might be missing a `bind-9.18` fix and a `bind-9.18-sub` rebase on top of `bind-9.18` is all we need. isc-projects/bind9!7705 is on the affected branch, tho.
```
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1:
#0 dns__catz_update_cb lib/dns/catz.c:2257:2 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#1 isc__nm_work_run lib/isc/netmgr/netmgr.c:3520:2 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#2 isc__trampoline_run lib/isc/trampoline.c:189:11 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#3 isc__nm_work_cb lib/isc/netmgr/netmgr.c:3532:9 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#4 uv__queue_work /usr/src/libuv-v1.44.1/src/threadpool.c:326:3 (BuildId: 120c450d14885aa5308bc95c4ea77de2c2b1cc36)
Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M1):
#0 attach lib/dns/rbtdb.c:788:11 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#1 dns_db_attach lib/dns/db.c:143:2 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#2 dns_catz_dbupdate_callback lib/dns/catz.c:2163:3 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#3 dns_db_endload lib/dns/db.c:293:3 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#4 zone_loaddone lib/dns/zone.c:17840:12 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#5 load_quantum lib/dns/master.c:3174:3 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#6 task_run lib/isc/task.c:815:5 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#7 isc_task_run lib/isc/task.c:896:10
#8 isc__nm_async_task lib/isc/netmgr/netmgr.c:848:11 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#9 process_netievent lib/isc/netmgr/netmgr.c (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#10 process_queue lib/isc/netmgr/netmgr.c:1013:8 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#11 process_all_queues lib/isc/netmgr/netmgr.c:767:25 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#12 async_cb lib/isc/netmgr/netmgr.c:796:6
#13 uv__async_io /usr/src/libuv-v1.44.1/src/unix/async.c:163:5 (BuildId: 120c450d14885aa5308bc95c4ea77de2c2b1cc36)
#14 isc__trampoline_run lib/isc/trampoline.c:189:11 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
Location is heap block of size 520 at 0x000000000019 allocated by thread T3:
#0 malloc <null> (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#1 mallocx lib/isc/./jemalloc_shim.h:35:10 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#2 mem_get lib/isc/mem.c:343:8
#3 isc__mem_get lib/isc/mem.c:761:8
#4 dns_catz_new_zone lib/dns/catz.c:833:9 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#5 dns_catz_add_zone lib/dns/catz.c:883:11 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#6 configure_catz_zone bin/named/server.c:3071:11 (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#7 configure_catz bin/named/server.c:3217:3
#8 configure_view bin/named/server.c:4345:3 (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#9 load_configuration bin/named/server.c:9570:3 (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#10 run_server bin/named/server.c:10307:2 (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#11 task_run lib/isc/task.c:815:5 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#12 isc_task_run lib/isc/task.c:896:10
#13 isc__nm_async_task lib/isc/netmgr/netmgr.c:848:11 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#14 process_netievent lib/isc/netmgr/netmgr.c (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#15 process_queue lib/isc/netmgr/netmgr.c:1013:8 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#16 process_all_queues lib/isc/netmgr/netmgr.c:767:25 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#17 async_cb lib/isc/netmgr/netmgr.c:796:6
#18 uv__async_io /usr/src/libuv-v1.44.1/src/unix/async.c:163:5 (BuildId: 120c450d14885aa5308bc95c4ea77de2c2b1cc36)
#19 isc__trampoline_run lib/isc/trampoline.c:189:11 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
Mutex M1 (0x000000000028) created at:
#0 pthread_mutex_init <null> (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#1 isc__mutex_init lib/isc/mutex.c:49:10 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#2 dns_catz_new_zones lib/dns/catz.c:798:2 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#3 configure_catz bin/named/server.c:3194:2 (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#4 configure_view bin/named/server.c:4345:3 (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#5 load_configuration bin/named/server.c:9570:3 (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#6 run_server bin/named/server.c:10307:2 (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#7 task_run lib/isc/task.c:815:5 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#8 isc_task_run lib/isc/task.c:896:10
#9 isc__nm_async_task lib/isc/netmgr/netmgr.c:848:11 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#10 process_netievent lib/isc/netmgr/netmgr.c (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#11 process_queue lib/isc/netmgr/netmgr.c:1013:8 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#12 process_all_queues lib/isc/netmgr/netmgr.c:767:25 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#13 async_cb lib/isc/netmgr/netmgr.c:796:6
#14 uv__async_io /usr/src/libuv-v1.44.1/src/unix/async.c:163:5 (BuildId: 120c450d14885aa5308bc95c4ea77de2c2b1cc36)
#15 isc__trampoline_run lib/isc/trampoline.c:189:11 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
Thread T3 (running) created by thread T3 at:
#0 pthread_create <null> (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#1 uv_thread_create_ex /usr/src/libuv-v1.44.1/src/unix/thread.c:279:9 (BuildId: 120c450d14885aa5308bc95c4ea77de2c2b1cc36)
#2 uv_once /usr/src/libuv-v1.44.1/src/unix/thread.c:440:7 (BuildId: 120c450d14885aa5308bc95c4ea77de2c2b1cc36)
#3 dns__catz_timer_cb lib/dns/catz.c:2118:2 (BuildId: 666f66a42faef3f08eba7d3dc612014f0ee091e0)
#4 task_run lib/isc/task.c:815:5 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#5 isc_task_run lib/isc/task.c:896:10
#6 isc__nm_async_task lib/isc/netmgr/netmgr.c:848:11 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#7 process_netievent lib/isc/netmgr/netmgr.c (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#8 process_queue lib/isc/netmgr/netmgr.c:1013:8 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#9 process_all_queues lib/isc/netmgr/netmgr.c:767:25 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#10 async_cb lib/isc/netmgr/netmgr.c:796:6
#11 uv__async_io /usr/src/libuv-v1.44.1/src/unix/async.c:163:5 (BuildId: 120c450d14885aa5308bc95c4ea77de2c2b1cc36)
#12 isc__trampoline_run lib/isc/trampoline.c:189:11 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
Thread T2 (running) created by main thread at:
#0 pthread_create <null> (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#1 isc_thread_create lib/isc/thread.c:73:8 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#2 isc__netmgr_create lib/isc/netmgr/netmgr.c:311:3 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#3 isc_managers_create lib/isc/managers.c:31:2 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#4 create_managers bin/named/main.c:1033:11 (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#5 setup bin/named/main.c:1304:11
#6 main bin/named/main.c:1576:2
Thread T3 (running) created by main thread at:
#0 pthread_create <null> (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#1 isc_thread_create lib/isc/thread.c:73:8 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#2 isc__netmgr_create lib/isc/netmgr/netmgr.c:311:3 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#3 isc_managers_create lib/isc/managers.c:31:2 (BuildId: ad8c165bb75f75ca44c4b2a82d508c708b57531d)
#4 create_managers bin/named/main.c:1033:11 (BuildId: 7e037db70ff852e5656e0d5112bfd951e2e76fbf)
#5 setup bin/named/main.c:1304:11
#6 main bin/named/main.c:1576:2
SUMMARY: ThreadSanitizer: data race lib/dns/catz.c:2257:2 in dns__catz_update_cb
```April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/3967linkfix in dnssec-guide/validation.rst2023-03-29T15:13:39ZMatthäus Wanderlinkfix in dnssec-guide/validation.rstThe doc file `doc/dnssec-guide/validation.rst` points to https://dnssec.vs.uni-due.de/, which is down. I'm the author of the website and relaunched the DNSSEC Resolver Test under https://wander.science/projects/dns/dnssec-resolver-test/
...The doc file `doc/dnssec-guide/validation.rst` points to https://dnssec.vs.uni-due.de/, which is down. I'm the author of the website and relaunched the DNSSEC Resolver Test under https://wander.science/projects/dns/dnssec-resolver-test/
I suggest to update the link.April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/28082.3.6 release checklist2023-05-04T09:00:46ZMarcin Godzina2.3.6 release checklist# Kea Release Checklist
This is thoroughly documented in [the Kea Release Process guide](https://wiki.isc.org/bin/view/QA/KeaReleaseProcess).
## Pre-Release Preparation
Some of those checks and updates can be made before the actual fr...# Kea Release Checklist
This is thoroughly documented in [the Kea Release Process guide](https://wiki.isc.org/bin/view/QA/KeaReleaseProcess).
## Pre-Release Preparation
Some of those checks and updates can be made before the actual freeze.
For new stable releases or maintenance releases, please don't use `kea-dev` build farm. Use dedicated build farm for each release cycle.
1. Check Jenkins results:
1. [x] Check Jenkins jobs for failures: [distcheck](https://jenkins.aws.isc.org/job/kea-dev/job/distcheck/), etc...
1. [x] Check [Jenkins Tests Report](https://jenkins.aws.isc.org/job/kea-dev/job/jenkins-tests-report/).
1. [x] Check [tarball check report](https://jenkins.aws.isc.org/job/kea-dev/job/build-tarball/Kea_20Build_20Checks/)
1. [x] Check [Performance Test Results](https://jenkins.isc.org/job/kea-dev/job/performance/KeaPerformanceReport/) in Jenkins for drops in performance.
1. Check versioning, ask the development team if:
- the library versions are being updated
- `KEA_HOOKS_VERSION` is being updated
- [x] create an issue for that for developers in Gitlab
- script: [./tools/bump-lib-versions.sh](https://gitlab.isc.org/isc-projects/kea/-/blob/master/tools/bump-lib-versions.sh) Kea-q.w.e Kea-a.b.c (where `a.b.c` is the version to be released and `q.w.e` is the version previous to that)
1. [x] Look at the issue numbers in commit descriptions. Add to ChangeLog a mention about any change with visible impact that had not been mentioned already.
1. ~~If any changes have been done to database schemas, then:~~
1. [ ] ~~Check that a previously released schema has not been changed.~~
1. [ ] ~~Check that the additions to `dhcpdb_create.*sql`, and nothing more nor less than what was added in this release, is present in a `upgrade_*_to_*.sh.in` script that should also have been added in this release.~~
1. Prepare Release Notes
1. [x] Create Release Notes on Kea GitLab wiki and notify @tomek about that. It should be created under "release notes" directory, like this one: https://gitlab.isc.org/isc-projects/kea/-/wikis/release%20notes/release-notes-2.1.0
1. [ ] Finish release notes and conduct its review. Also please notify @sgoldlust or @vicky that release notes are ready for review.
1. [x] Run [release-upload-to-cloudsmith](https://jenkins.aws.isc.org/job/kea-dev/job/release-upload-to-cloudsmith/) as running parameter `TarballOrPkg` select `packages` and [release-pkgs-check](https://jenkins.aws.isc.org/job/kea-dev/job/release-pkgs-check/) to test repositories for correctness.
1. ~~If a new Cloudsmith repository is used, then:~~
1. [ ] ~~Make sure freeradius packages are uploaded to the Cloudsmith repository or copied from a previous repository.~~
1. [ ] ~~Make sure access tokens have been synchronized from previous Cloudsmith repositories and to the [check-pkgs.py](https://gitlab.isc.org/isc-private/qa-dhcp/-/blob/master/kea/pkgs-check/check-pkgs.py) QA tool.~~
1. [x] Check if ReadTheDocs can build Kea documentation.
1. Trigger rebuilding docs on [readthedocs.org](https://readthedocs.org/projects/kea/builds) and wait for the build to complete.
The following steps may involve changing files in the repository.
1. [x] Run [update-code-for-release.py](https://gitlab.isc.org/isc-private/qa-dhcp/-/blob/master/kea/build/update-code-for-release.py) <br>
Example command: `GITLAB_TOKEN='...' ./update-code-for-release.py 1.9.7 --repo-dir ~/isc/repos/kea/` Use `--upload` to commit changes. <br>
Help: `GITLAB_TOKEN="..." ./update-code-for-release.py --help`<br>
This script makes the following changes and actions:
1. run prepare_kea_release.sh that does:
1. add release entries in ChangeLogs
1. update Kea version in configure.ac
1. update copyright years in files that were changed in current year
1. sort message files
1. regenerate message files headers
2. regenerate parsers using Bison from Docker<br>
With `--upload`:
3. create an issue in GitLab for release changes in kea repo
4. create branches and merge requests for kea and kea-premium
5. commit the changes in both repos
6. checkout created branches in both repos
7. commit and push the changes to GitLab server
1. Check manually User's Guide sections:
1. Chapter 1. Introduction
1. [x] On what platforms we are running tests using Jenkins? Update Supported Platforms in platforms.rst file.
1. [x] Did we add any additional 3rd party software? Update if needed
1. [x] Is there a new tool installed in bin or sbin released this time? If yes, is it documented?
1. Chapter 2. Quick Start
1. [x] Has the default installation process changed (for kea and hooks)? If yes, are those changes documented and highlighted in the release notes?
1. Chapter 3. Installation
1. [x] Check installation hierarchy (this is also automatically checked at the end of [ut-extended job](https://jenkins.aws.isc.org/job/kea-dev/job/ut-extended/))
1. [x] Check and update Build Requirements
1. [x] Check configure options against what `./configure -h` says
1. [x] Check ChangeLog entries in Kea main and premium: spelling, trailing whitespaces, etc.
1. [x] Check AUTHORS, INSTALL, README files in Kea main and premium.
- AUTHORS: update credits
- README: check "provides" with Release Notes, User Guide (1.3 Kea Software)
1. [x] If changes were made, commit the change, push the branch to the main repository and request a review. Once the changes have been approved, merge the MR to master.
## Build selection, tarballs upload and sanity checks
This is the last moment to freeze code! :snowflake:
1. [x] Go to [build-tarball](https://jenkins.aws.isc.org/job/kea-dev/job/build-tarball/) Jenkins job and pick the last tarball built - it will be a release candidate.
1. [x] Check tarball before requesting sanity checks from the development team.
1. Download tarballs from picked Jenkins build
1. Check hook libraries.
1. Are there any new hook libraries installed in this release?
1. Are they in the proper tarball? Premium or subscription?
1. Do they have their own package?
1. Check sizes - is the new package reasonable?
1. Check installation tree, compare it with the previous release
1. Check installed libraries.
1. which were updated? (save results)
1. Do any of the libraries from the current release have lower version than in the previous release?
1. Uninstall Kea, check what left (there should be just configuration files)
1. Check if all of the installed binaries has man page
1. if not, is it in the tarball?
1. are man page up-to-date?
1. Check if documentation is properly formatted, has correct versions and dates.
1. it's advised to search for previous version numbers, some of them are statically added in statements that are no longer valid
1. [x] Upload tarballs to repo.isc.org using Jenkins and send sanity checks request.
1. Go to [release-tarball-upload](https://jenkins.aws.isc.org/job/kea-dev/job/release-tarball-upload/) Jenkins job.
1. Click "Build with Parameters"
1. In field "Tarball" select picked tarball build
1. In field "Release_Candidate" pick:
1. rc1 if this is the first selected build for release, it will push the selected tarballs to repo.isc.org, to a directory suffixed with indicated rc#
1. next rc# if this is a respin after some fixes (note: it is not possible to pick previous rc number - it will result in an error)
1. Submit the job that will automatically:
1. Upload the tarballs <br>
and if this is not the final version:
1. Create a GitLab issue for sanity checks, put there the announcement
1. Send Sanity Checks announcement via email to dhcp-team@isc.org and to DHCP channel on Mattermost.<br>
The announcement includes:
- a link to chapter 4 Sanity Checks of the release process: [KeaReleaseProcess - SanityChecks](https://wiki.isc.org/bin/view/QA/KeaReleaseProcess#4.%20Sanity%20Checks)
- a link to the GitLab issue
- tarballs locations with SHA256 checksums
- rpm/deb packages locations and versions
## Releasing Tarballs and Packages
1. [x] Update Release Notes with ChangeLog entries
1. [x] Mark Jenkins jobs with release artifacts to be kept forever and update description of build by adding there version of released kea (e.g. Kea-2.2.2): <br>
Go to the following Jenkins jobs, click release build and then, on the build page, click `Keep this build forever` button and edit description: <br>
1. [build-tarball](https://jenkins.aws.isc.org/job/kea-dev/job/build-tarball/)
1. [pkg job](https://jenkins.aws.isc.org/job/kea-dev/job/pkg/)
1. [x] Upload final tarballs to repo.isc.org
1. Go to [release-tarball-upload](https://jenkins.aws.isc.org/job/kea-dev/job/release-tarball-upload/) Jenkins job.
1. Click "Build with Parameters"
1. In field "Tarball" select picked tarball build
1. In field "Release_Candidate" pick final <br>
This job will also:
- open an issue on [the signing repository](https://gitlab.isc.org/isc-private/signing/-/issues) for signing final tarballs on repo.isc.org
- create Git tags `Kea-a.b.c` in Kea main and premium repositories
- if release engineer is holding personal signing key, please use [sign, verify, and upload script](https://gitlab.isc.org/isc-private/qa-dhcp/-/blob/master/kea/build/sign_kea_and_upload_asc.sh)
- if release enginner do NOT have signing key, please contact team member.
1. [x] Upload final RPM & DEB packages, tarballs and sign files to cloudsmith.io
1. Go to [release-upload-to-cloudsmith](https://jenkins.aws.isc.org/job/kea-dev/job/release-upload-to-cloudsmith/).
1. Click "Build with Parameters" link
1. Pick your selected pkg build in Packages field, and select `PrivPubRepos: "both"`, `TarballOrPkg: "both"`, `TestProdRepos: "production"` and click `Build` button.
- this step is also veryfing sign files
1. When it finishes run check: [releases-pkgs-check](https://jenkins.aws.isc.org/job/kea-dev/job/release-pkgs-check/).
1. [x] Update ReadTheDocs
1. Trigger rebuilding docs on [readthedocs.org](https://readthedocs.org/projects/kea/builds).
1. Publish currently released version. On the `Versions` tab, scroll down to `Activate a version`, search for `kea-a.b.c` and click `Activate`.
1. For stable releases, change the default version to point to this stable release.
1. [x] Create an issue and a merge request to bump up Kea version in `configure.ac` to next development version which could be, based on just released version `a.b.c`:
* `a.b.z-git` where `z == c + 1` or
* `a.y.0-git` where `y == b + 1` or
* `x.1.0-git` where `x == a + 1`
1. [x] Send a request for publishing the release on the Support Mattermost channel linking the Signing issue and the release checklist issue.
### On the Day of Public Release
- [ ] ***(Support)*** Wait for clearance from Security Officer to proceed with the public release (if applicable).
- [ ] ***(Support)*** Confirm that the tarballs have the checksums mentioned on the signing ticket.
- [ ] ***(Support)*** Place tarballs in public location on FTP site.
- [ ] ***(Support)*** Publish links to downloads on ISC website.
- [ ] ***(Support)*** Write release email to *kea-announce*.
- [ ] ***(Support)*** Write email to *kea-users* (if a major release).
- [ ] ***(Support)*** Send eligible customers updated links to the Subscription software FTP site.
- [ ] ***(Support)*** If it is a new `major.minor` version, SWENG will have created a new repo in Cloudsmith, which will need the customer tokens migrated from an existing repo. Then update support customers that this new private repo exists.
- [ ] ***(Support)*** Update tickets in case of waiting for support customers.
- [x] ***(Support)*** Inform Marketing of the release.
- [ ] ***(Marketing)*** If a new Cloudsmith repository is used, update the Zapier scripts.
- [x] ***(Marketing)*** Upload Premium hooks tarball to SendOwl. Create a new product if a new branch, otherwise update existing product. Send notifications to existing subscribers of the new version.
- [x] ***(Marketing)*** Announce on social media.
- [x] ***(Marketing)*** Update [Wikipedia entry for Kea](https://en.wikipedia.org/wiki/Kea_(software)).
- [ ] ***(Marketing)*** Write blog article (if a major release).
- [ ] ***(Marketing)*** Update [Kea page on web site if any new hooks](https://www.isc.org/kea/).
- [ ] ***(Marketing)*** Update Kea Premium and Kea Subscription data sheets if any new hooks.
- [ ] ***(Marketing)*** Update [significant features matrix](https://kb.isc.org/docs/en/aa-01615) (if any significant new features).
- [ ] ***(Marketing)*** Update [Kea documentation page in KB](https://kb.isc.org/docs/en/kea-administrator-reference-manual).kea2.3.6Marcin GodzinaMarcin Godzinahttps://gitlab.isc.org/isc-projects/bind9/-/issues/3966Connect callback calling bug in TLS stream2023-04-11T17:42:26ZOndřej SurýConnect callback calling bug in TLS streamWhile working on c6d215b1ec1, a new ThreadSanitizer error and crash has happened:
```
==================
WARNING: ThreadSanitizer: heap-use-after-free (pid=3475979)
Read of size 4 at 0x7b4c001c1500 by thread T291:
#0 isc__nmhandle_...While working on c6d215b1ec1, a new ThreadSanitizer error and crash has happened:
```
==================
WARNING: ThreadSanitizer: heap-use-after-free (pid=3475979)
Read of size 4 at 0x7b4c001c1500 by thread T291:
#0 isc__nmhandle_detach /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1035:2 (libisc-9.19.12-dev.so+0x1ee74) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#1 tls_senddone /home/ondrej/Projects/bind9/lib/isc/netmgr/tlsstream.c:191:3 (libisc-9.19.12-dev.so+0x40bed) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#2 isc___nm_sendcb /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1910:2 (libisc-9.19.12-dev.so+0x249cb) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#3 <null> <null> (libuv.so.1+0x168a0) (BuildId: 7f7f8c148150666c7b116bf98bf6e27f96c697a9)
#4 loop_thread /home/ondrej/Projects/bind9/lib/isc/loop.c:299:2 (libisc-9.19.12-dev.so+0x743e0) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#5 isc__trampoline_run /home/ondrej/Projects/bind9/lib/isc/trampoline.c:202:11 (libisc-9.19.12-dev.so+0xa7cce) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
Previous write of size 8 at 0x7b4c001c1500 by thread T291:
#0 free <null> (tls_test+0x44e285) (BuildId: 0a4db467257a69fefc960e3385738e8f478c2a73)
#1 sdallocx /home/ondrej/Projects/bind9/lib/isc/./jemalloc_shim.h:80:2 (libisc-9.19.12-dev.so+0x803fa) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#2 mem_put /home/ondrej/Projects/bind9/lib/isc/mem.c:326:2 (libisc-9.19.12-dev.so+0x79ca6) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#3 isc__mem_put /home/ondrej/Projects/bind9/lib/isc/mem.c:684:2 (libisc-9.19.12-dev.so+0x7ac23) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#4 nmhandle_free /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:972:2 (libisc-9.19.12-dev.so+0x2818f) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#5 nmhandle_destroy /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1008:2 (libisc-9.19.12-dev.so+0x1f5d2) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#6 isc__nmhandle_detach /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1047:3 (libisc-9.19.12-dev.so+0x1efe1) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#7 connect_send_cb /home/ondrej/Projects/bind9/tests/isc/netmgr_common.c:293:2 (tls_test+0x4d8b80) (BuildId: 0a4db467257a69fefc960e3385738e8f478c2a73)
#8 tls_senddone /home/ondrej/Projects/bind9/lib/isc/netmgr/tlsstream.c:190:3 (libisc-9.19.12-dev.so+0x40be4) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#9 isc___nm_sendcb /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1910:2 (libisc-9.19.12-dev.so+0x249cb) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#10 <null> <null> (libuv.so.1+0x168a0) (BuildId: 7f7f8c148150666c7b116bf98bf6e27f96c697a9)
#11 loop_thread /home/ondrej/Projects/bind9/lib/isc/loop.c:299:2 (libisc-9.19.12-dev.so+0x743e0) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
Thread T291 'isc-loop-0003' (tid=3476621, running) created by main thread at:
#0 pthread_create <null> (tls_test+0x44f85b) (BuildId: 0a4db467257a69fefc960e3385738e8f478c2a73)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/thread.c:70:8 (libisc-9.19.12-dev.so+0x9c00e) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#2 isc_loopmgr_run /home/ondrej/Projects/bind9/lib/isc/loop.c:478:3 (libisc-9.19.12-dev.so+0x741d4) (BuildId: a03629bc4ef7d953712229d3b234a56784137779)
#3 run_test_tls_recv_send_quota /home/ondrej/Projects/bind9/tests/isc/tls_test.c:88:1 (tls_test+0x4d7547) (BuildId: 0a4db467257a69fefc960e3385738e8f478c2a73)
SUMMARY: ThreadSanitizer: heap-use-after-free /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1035:2 in isc__nmhandle_detach
==================
```
I've uploaded the `systemctl dump` to: `ondrej@bikeshed.isc.org:/data/shared/sweng/ondrej`April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)Artem BoldarievArtem Boldarievhttps://gitlab.isc.org/isc-projects/bind9/-/issues/3965Value stored to 'source' during its initialization is never read in lib/dns/m...2023-04-11T17:28:12ZMichal NowakValue stored to 'source' during its initialization is never read in lib/dns/master.cscan-build v16 from isc-projects/bind9!7721 [fails](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3266336).
```
File: master.c
Warning: line 1046, column 14
Value stored to 'source' during its initialization is never read
```
[scan ...scan-build v16 from isc-projects/bind9!7721 [fails](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3266336).
```
File: master.c
Warning: line 1046, column 14
Value stored to 'source' during its initialization is never read
```
[scan build v16 report](/uploads/3ae17ba708ba1a38b30f5fbdd9d19c86/artifacts-174.zip)April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/39629.18.13 ARM formatting errors2023-04-05T12:44:58ZEverett Fulton9.18.13 ARM formatting errors(Quote from a non-customer email to the ISC support team)
I find two errore on Bv9ARM.
The first is:
```
$ diff -cr zones.inc.rst_orig zones.inc.rst
*** zones.inc.rst_orig 2023-03-06 17:04:58.759796386 +0800
--- zones.inc.rst 2...(Quote from a non-customer email to the ISC support team)
I find two errore on Bv9ARM.
The first is:
```
$ diff -cr zones.inc.rst_orig zones.inc.rst
*** zones.inc.rst_orig 2023-03-06 17:04:58.759796386 +0800
--- zones.inc.rst 2023-03-18 09:55:18.865163982 +0800
***************
*** 460,465 ****
--- 460,466 ----
is equivalent to:
::
+
HOST-0000.EXAMPLE. A 1.2.3.1
HOST-0001.EXAMPLE. A 1.2.3.2
HOST-0002.EXAMPLE. A 1.2.3.3
```
the second is at the page 233 of the pdf version,
8.2.33 zone Block Grammar,
in the line after zone, Grammar:
there are two zone keywords in this line,
`zone zone <string> [ <class> ] {`April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)Tom KrizekTom Krizekhttps://gitlab.isc.org/isc-projects/bind9/-/issues/3960Invalid handling of ISC_R_SHUTTINGDOWN in resquery_response2023-04-11T21:05:03ZOndřej SurýInvalid handling of ISC_R_SHUTTINGDOWN in resquery_response```
#0 rctx_respinit (query=query@entry=0x7fab98271600, fctx=fctx@entry=0x7fab98274000, result=result@entry=ISC_R_SHUTTINGDOWN, region=region@entry=0x0, rctx=rctx@entry=0x7fab99ffdf20) at resolver.c:7917
#1 0x00007fabbaf2c74f in resque...```
#0 rctx_respinit (query=query@entry=0x7fab98271600, fctx=fctx@entry=0x7fab98274000, result=result@entry=ISC_R_SHUTTINGDOWN, region=region@entry=0x0, rctx=rctx@entry=0x7fab99ffdf20) at resolver.c:7917
#1 0x00007fabbaf2c74f in resquery_response (eresult=ISC_R_SHUTTINGDOWN, region=0x0, arg=0x7fab98271600) at resolver.c:7498
#2 0x00007fabbae4d610 in udp_dispentry_cancel (resp=0x7fab9820a000, result=result@entry=ISC_R_SHUTTINGDOWN) at dispatch.c:1666
#3 0x00007fabbae50c73 in dispentry_cancel (resp=<optimized out>, result=result@entry=ISC_R_SHUTTINGDOWN) at dispatch.c:1782
#4 0x00007fabbae5179e in send_done (handle=<optimized out>, result=ISC_R_SHUTTINGDOWN, cbarg=<optimized out>) at dispatch.c:2108
#5 0x00007fabbb2207f7 in isc__nm_sendcb_job (arg=<optimized out>) at netmgr/netmgr.c:2058
#6 0x00007fabbb238f49 in isc__job_cb (idle=0x7fabb4925c88) at job.c:75
#7 0x00007fabbb1728a1 in ?? () from /lib/x86_64-linux-gnu/libuv.so.1
#8 0x00007fabbb16b9b9 in uv_run () from /lib/x86_64-linux-gnu/libuv.so.1
#9 0x00007fabbb23f718 in loop_run (loop=loop@entry=0x7fabb76c66f0) at loop.c:273
#10 0x00007fabbb23f7b8 in loop_thread (arg=0x7fabb76c66f0) at loop.c:299
#11 0x00007fabbb254b27 in isc__trampoline_run (arg=0x56329f853810) at trampoline.c:202
#12 0x00007fabba4fbfd4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#13 0x00007fabba57c66c in ?? () from /lib/x86_64-linux-gnu/libc.so.6
```
The `region` is `NULL` when `eresult` is `ISC_R_SHUTTINGDOWN`, but `rctx_respinit` dereferences `region`.April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)https://gitlab.isc.org/isc-projects/bind9/-/issues/3955Assertion failure in dns__catz_zone_destroy() during shutdown2023-04-20T10:54:36ZArаm SаrgsyаnAssertion failure in dns__catz_zone_destroy() during shutdownSee https://gitlab.isc.org/isc-projects/bind9/-/jobs/3251511
```
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f4ff83a4e5c in __pthread_kill_implementation () from /lib64/libc.so.6
[Current thread is 1 (Thread 0x7f4ff4fbd6...See https://gitlab.isc.org/isc-projects/bind9/-/jobs/3251511
```
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f4ff83a4e5c in __pthread_kill_implementation () from /lib64/libc.so.6
[Current thread is 1 (Thread 0x7f4ff4fbd6c0 (LWP 27104))]
#0 0x00007f4ff83a4e5c in __pthread_kill_implementation () from /lib64/libc.so.6
#1 0x00007f4ff8354a76 in raise () from /lib64/libc.so.6
#2 0x00007f4ff833e7fc in abort () from /lib64/libc.so.6
#3 0x00007f4ff95d8f80 in abort () from /lib64/libtsan.so.2
#4 0x0000000000424f48 in assertion_failed (file=<optimized out>, line=<optimized out>, type=<optimized out>, cond=<optimized out>) at main.c:236
#5 0x00007f4ff90012d7 in isc_assertion_failed (file=file@entry=0x7f4ff8f4f2b7 "catz.c", line=line@entry=1004, type=type@entry=isc_assertiontype_insist, cond=cond@entry=0x7f4ff8f4f401 "!catz->updaterunning") at assertions.c:48
#6 0x00007f4ff8d3c971 in dns__catz_zone_destroy (catz=0x7b5400010180) at catz.c:1004
#7 dns_catz_zone_unref (ptr=ptr@entry=0x7b5400010180) at catz.c:1063
#8 0x00007f4ff8d3ca30 in dns_catz_zone_detach (ptrp=ptrp@entry=0x7f4ff4ef95e0) at catz.c:1063
#9 0x00007f4ff8d3f113 in dns__catz_shutdown (catz=<optimized out>) at catz.c:944
#10 dns_catz_shutdown_catzs (catzs=0x7b2000014b00) at catz.c:1049
#11 0x00007f4ff8ed5c7d in view_flushanddetach (viewp=viewp@entry=0x7f4ff4ef96f8, flush=flush@entry=false) at view.c:684
#12 0x00007f4ff8ed5e86 in dns_view_detach (viewp=viewp@entry=0x7f4ff4ef96f8) at view.c:716
#13 0x0000000000453cef in shutdown_server (task=<optimized out>, event=<optimized out>) at server.c:10137
#14 0x00007f4ff9030573 in task_run (task=0x7b3000000900) at task.c:815
#15 isc_task_run (task=0x7b3000000900) at task.c:896
#16 0x00007f4ff8fdbdec in isc__nm_async_task (worker=worker@entry=0x7ba400000000, ev0=ev0@entry=0x7b4800001980) at netmgr/netmgr.c:848
#17 0x00007f4ff8fe70ab in process_netievent (worker=worker@entry=0x7ba400000000, ievent=ievent@entry=0x7b4800001980) at netmgr/netmgr.c:920
#18 0x00007f4ff8fe7d7c in process_queue (worker=worker@entry=0x7ba400000000, type=type@entry=NETIEVENT_TASK) at netmgr/netmgr.c:1013
#19 0x00007f4ff8fe8af2 in process_all_queues (worker=0x7ba400000000) at netmgr/netmgr.c:767
#20 async_cb (handle=0x7ba400000360) at netmgr/netmgr.c:796
#21 0x00007f4ff86da18f in uv__async_io (loop=0x7ba400000010, w=0x7ba4000001d8, events=1) at /usr/src/libuv-v1.44.1/src/unix/async.c:163
#22 0x00007f4ff86f65da in uv__io_poll (loop=0x7ba400000010, timeout=-1) at /usr/src/libuv-v1.44.1/src/unix/epoll.c:374
#23 0x00007f4ff86dac2d in uv_run (loop=0x7ba400000010, mode=UV_RUN_DEFAULT) at /usr/src/libuv-v1.44.1/src/unix/core.c:391
#24 0x00007f4ff8fe818f in nm_thread (worker0=0x7ba400000000) at netmgr/netmgr.c:698
#25 0x00007f4ff903c0c2 in isc__trampoline_run (arg=0x7b0c00002190) at trampoline.c:189
#26 0x00007f4ff95b13f0 in __tsan_thread_start_func () from /lib64/libtsan.so.2
#27 0x00007f4ff83a312d in start_thread () from /lib64/libc.so.6
#28 0x00007f4ff8423d74 in clone () from /lib64/libc.so.6
```
Assertion on `!catz->updaterunning` in `dns__catz_zone_destroy()`, i.e. the catalog zone has been destroyed during shutdown while the update process was still running.
Holding an additional reference to `catz`, while performing the update process, should fix this issue.April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/3953Deprecate and remove (root-)delegation-only2023-04-11T17:19:54ZOndřej SurýDeprecate and remove (root-)delegation-onlyThe `delegation-only` and `root-delegation-only` options were introduced as a response to the "SiteFinder" incident, as it can be seen [here](https://circleid.com/posts/the_name_domain_disrupted_by_site_finder_patch) and [here](https://w...The `delegation-only` and `root-delegation-only` options were introduced as a response to the "SiteFinder" incident, as it can be seen [here](https://circleid.com/posts/the_name_domain_disrupted_by_site_finder_patch) and [here](https://www.afnic.fr/en/observatory-and-resources/news/warning-for-bind-and-delegation-only-users/) this created more problems than it solved and the options should be deprecated and removed; possibly in the expedited way.April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)https://gitlab.isc.org/isc-projects/bind9/-/issues/3950Unexpected NODATA answers instead of successful response or SERVFAIL with ser...2023-10-11T13:10:49ZMaksym OdinintsevUnexpected NODATA answers instead of successful response or SERVFAIL with serve-stale answers enabled and serve-stale-client-timeout set to positive value<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confident...<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confidential!
-->
### Summary
Unexpected NODATA answer with serve-stale answers enabled and `serve-stale-client-timeout` set more that 0.
### BIND version used
```
BIND 9.16.38-Ubuntu (Extended Support Version) <id:af0056a>
running on Linux x86_64 5.15.0-1028-aws #32~20.04.1-Ubuntu SMP Mon Jan 9 18:02:08 UTC 2023
built by make with '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/x86_64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--enable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=/usr' '--with-libidn2' '--with-json-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' '--enable-dnstap' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fdebug-prefix-map=/build/bind9-5oQ5lS/bind9-9.16.38=. -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
compiled by GCC 9.4.0
compiled with OpenSSL version: OpenSSL 1.1.1f 31 Mar 2020
linked to OpenSSL version: OpenSSL 1.1.1f 31 Mar 2020
compiled with libuv version: 1.44.2
linked to libuv version: 1.44.2
compiled with libxml2 version: 2.9.10
linked to libxml2 version: 20910
compiled with json-c version: 0.13.1
linked to json-c version: 0.13.1
compiled with zlib version: 1.2.11
linked to zlib version: 1.2.11
linked to maxminddb version: 1.4.2
compiled with protobuf-c version: 1.3.3
linked to protobuf-c version: 1.3.3
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): yes
default paths:
named configuration: /etc/bind/named.conf
rndc configuration: /etc/bind/rndc.conf
DNSSEC root key: /etc/bind/bind.keys
nsupdate session key: //run/named/session.key
named PID file: //run/named/named.pid
named lock file: //run/named/named.lock
geoip-directory: /usr/share/GeoIP
```
### Steps to reproduce
There are two cases affected with unexpected NODATA answers.
1. First test case: recursion enabled, upstreams available, no data in cache.
Expected response: NOERROR with data if recursion succeeded in timeout value (10s)
- OS: Ubuntu 20.04, kernel 5.15.0-1028-aws
- Freshly installed Bind9 from https://launchpad.net/~isc/+archive/ubuntu/bind-esv
- In config file set that options:
```
dnssec-validation no;
stale-cache-enable yes;
stale-answer-enable yes;
stale-answer-client-timeout 150;
qname-minimization disabled;
listen-on-v6 { none; };
```
- Restart named (or do `rndc flush` to clean up caches)
- run command: `dig +timeout=10 @127.0.0.1 ta1.myctl.com A`
If time to answer takes more than 150ms and there is no data in cache, I get NOERROR NODATA answer exact after 150ms:
```
root@ip-172-31-40-214:/etc/bind# dig +timeout=10 @127.0.0.1 ta1.myctl.com A
; <<>> DiG 9.16.38-Ubuntu <<>> +timeout @127.0.0.1 ta1.myctl.com A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1739
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 6
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: d202b375fc315c390100000064134c5002732d2be0d5cfc3 (good)
;; QUESTION SECTION:
;ta1.myctl.com. IN A
;; AUTHORITY SECTION:
myctl.com. 172800 IN NS ns-canada.topdns.com.
myctl.com. 172800 IN NS ns-usa.topdns.com.
myctl.com. 172800 IN NS ns-uk.topdns.com.
;; ADDITIONAL SECTION:
ns-uk.topdns.com. 172800 IN A 77.247.183.137
ns-uk.topdns.com. 172800 IN A 108.61.150.91
ns-usa.topdns.com. 172800 IN A 46.166.189.99
ns-usa.topdns.com. 172800 IN A 108.61.12.163
ns-canada.topdns.com. 172800 IN A 109.201.142.225
;; Query time: 151 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Mar 16 17:05:20 UTC 2023
;; MSG SIZE rcvd: 225
```
I expect successful answer if upstreams answer within dig timeout option value.
I couldn't reproduce such behaviour with `qname-minimization relaxed;`
As soon as data appears in cache, I see correct answers.
2. Second test case: recursion enabled, upstreams unavailable, no data in cache.
Expected response: SERVFAIL instead of NODATA
The same initial OS and Bind version like in the first test case:
- asking query to fill prime cache: dig +timeout=10 @127.0.0.1 ta1.myctl.com A
- block all external DNS answers with iptables:
```
sudo iptables -A INPUT -s 127.0.0.0/24 -p udp --sport 53 -j ACCEPT && \
sudo iptables -A INPUT -s 127.0.0.0/24 -p tcp --sport 53 -j ACCEPT && \
sudo iptables -A INPUT -s 0.0.0.0/0 -p udp --sport 53 -j DROP && \
sudo iptables -A INPUT -s 0.0.0.0/0 -p tcp --sport 53 -j DROP
```
- flushing name `ta1.myctl.com` with command: `rndc flushname ta1.myctl.com`
- running query: `dig +timeout=10 @127.0.0.1 ta1.myctl.com A`
Getting NODATA response after 150ms:
```
; <<>> DiG 9.16.38-Ubuntu <<>> +timeout @127.0.0.1 ta1.myctl.com A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54375
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 6
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 6373254ddb0b95f10100000064134f9530da2cc66aff4cd7 (good)
;; QUESTION SECTION:
;ta1.myctl.com. IN A
;; AUTHORITY SECTION:
myctl.com. 3556 IN NS ns-canada.topdns.com.
myctl.com. 3556 IN NS ns-usa.topdns.com.
myctl.com. 3556 IN NS ns-uk.topdns.com.
;; ADDITIONAL SECTION:
ns-uk.topdns.com. 172756 IN A 77.247.183.137
ns-uk.topdns.com. 172756 IN A 108.61.150.91
ns-usa.topdns.com. 172756 IN A 46.166.189.99
ns-usa.topdns.com. 172756 IN A 108.61.12.163
ns-canada.topdns.com. 172756 IN A 109.201.142.225
;; Query time: 151 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Mar 16 17:19:17 UTC 2023
;; MSG SIZE rcvd: 225
```
- running the same query again:
Getting SERVFAIL (after `resolver-query-timeout`)
```
; <<>> DiG 9.16.38-Ubuntu <<>> +timeout @127.0.0.1 ta1.myctl.com A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 25802
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 23fed5e0a2a249db01000000641350293043165904e30895 (good)
;; QUESTION SECTION:
;ta1.myctl.com. IN A
;; Query time: 9827 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Mar 16 17:21:45 UTC 2023
;; MSG SIZE rcvd: 70
```
### What is the current *bug* behavior?
When serve-stale answers is enabled, and stale-answer-client-timeout set to positive value, I see unexpected NODATA answers (exact after stale-answer-client-timeout passed) instead of success or SERVFAIL.
### What is the expected *correct* behavior?
First test case: When upstreams available and can answer in timeout option (in all examples it is 10 seconds), and data is not in cache - return answer.
Second test case: When upstreams are not available and no data in cache - return SERVFAIL instead of NODATA
### Relevant configuration files
```
logging {
channel "standard_var_log" {
file "/var/log/named/named.log" versions 3 size 104857600;
severity info;
print-time yes;
print-severity yes;
print-category yes;
};
category "default" {
"standard_var_log";
};
category "lame-servers" {
"null";
};
category "rpz" {
"null";
};
category "serve-stale" {
"standard_var_log";
};
};
options {
directory "/var/cache/bind";
listen-on-v6 {
"none";
};
dnssec-validation no;
qname-minimization disabled;
stale-answer-enable yes;
stale-answer-client-timeout 150;
stale-cache-enable yes;
};
zone "." {
type hint;
file "/usr/share/dns/root.hints";
};
zone "localhost" {
type master;
file "/etc/bind/db.local";
};
zone "127.in-addr.arpa" {
type master;
file "/etc/bind/db.127";
};
zone "0.in-addr.arpa" {
type master;
file "/etc/bind/db.0";
};
zone "255.in-addr.arpa" {
type master;
file "/etc/bind/db.255";
};
```
### Relevant logs and/or screenshots
```
16-Mar-2023 17:18:37.566 general: info: received control channel command 'flushname ta1.myctl.com'
16-Mar-2023 17:18:37.566 general: info: flushing name 'ta1.myctl.com' in all cache views succeeded
16-Mar-2023 17:18:37.586 general: info: received control channel command 'dumpdb'
16-Mar-2023 17:18:37.586 general: info: dumpdb started
16-Mar-2023 17:18:37.590 general: info: dumpdb complete
16-Mar-2023 17:18:38.846 general: info: received control channel command 'flushname ta1.myctl.com'
16-Mar-2023 17:18:38.846 general: info: flushing name 'ta1.myctl.com' in all cache views succeeded
16-Mar-2023 17:18:38.866 general: info: received control channel command 'dumpdb'
16-Mar-2023 17:18:38.866 general: info: dumpdb started
16-Mar-2023 17:18:38.870 general: info: dumpdb complete
16-Mar-2023 17:18:40.506 general: info: received control channel command 'flushname ta1.myctl.com'
16-Mar-2023 17:18:40.506 general: info: flushing name 'ta1.myctl.com' in all cache views succeeded
16-Mar-2023 17:18:40.526 general: info: received control channel command 'dumpdb'
16-Mar-2023 17:18:40.526 general: info: dumpdb started
16-Mar-2023 17:18:40.530 general: info: dumpdb complete
16-Mar-2023 17:19:16.978 general: info: received control channel command 'flushname ta1.myctl.com'
16-Mar-2023 17:19:16.982 general: info: flushing name 'ta1.myctl.com' in all cache views succeeded
16-Mar-2023 17:19:16.998 general: info: received control channel command 'dumpdb'
16-Mar-2023 17:19:16.998 general: info: dumpdb started
16-Mar-2023 17:19:17.022 general: info: dumpdb complete
16-Mar-2023 17:19:19.394 general: info: received control channel command 'flushname ta1.myctl.com'
16-Mar-2023 17:19:19.394 general: info: flushing name 'ta1.myctl.com' in all cache views succeeded
16-Mar-2023 17:19:19.414 general: info: received control channel command 'dumpdb'
16-Mar-2023 17:19:19.414 general: info: dumpdb started
16-Mar-2023 17:19:19.422 general: info: dumpdb complete
16-Mar-2023 17:19:27.022 client: info: client @0x7f8fc85d8880 127.0.0.1#49750 (ta1.myctl.com): recursion loop detected
16-Mar-2023 17:19:27.022 client: info: client @0x7f8fc85e97a0 127.0.0.1#54607 (ta1.myctl.com): recursion loop detected
16-Mar-2023 17:21:35.554 general: info: received control channel command 'flushname ta1.myctl.com'
16-Mar-2023 17:21:35.554 general: info: flushing name 'ta1.myctl.com' in all cache views succeeded
16-Mar-2023 17:21:35.570 general: info: received control channel command 'dumpdb'
16-Mar-2023 17:21:35.570 general: info: dumpdb started
16-Mar-2023 17:21:35.578 general: info: dumpdb complete
16-Mar-2023 17:21:45.594 client: info: client @0x7f8fc85e97a0 127.0.0.1#38227 (ta1.myctl.com): recursion loop detected
16-Mar-2023 17:21:45.594 client: info: client @0x7f8fc85d8880 127.0.0.1#41798 (ta1.myctl.com): recursion loop detected
```
Reference: [Support ticket #21952](https://support.isc.org/Ticket/Display.html?id=21952)June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/3939Insufficient information to identify key2023-04-11T16:57:13ZMark AndrewsInsufficient information to identify keyWe currently log lines like the following which really should have the name of the key as context is not always enough to identify.
```
13-Mar-2023 12:10:33.312 Removing expired key 1556/ECDSAP256SHA256 from DNSKEY RRset.
```We currently log lines like the following which really should have the name of the key as context is not always enough to identify.
```
13-Mar-2023 12:10:33.312 Removing expired key 1556/ECDSAP256SHA256 from DNSKEY RRset.
```April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)https://gitlab.isc.org/isc-projects/bind9/-/issues/3937NSEC records aren't signed with both configured algorithms during NSEC3->NSEC...2023-04-11T17:02:11ZTom KrizekNSEC records aren't signed with both configured algorithms during NSEC3->NSEC transitionThis behavior has been observed in `nsec3` system test: https://gitlab.isc.org/isc-projects/bind9/-/jobs/3223579. It seems to be an intermittent failure that isn't reliably reproducible.
```
I:nsec3:verify DNSSEC for zone nsec3-to-rsasha...This behavior has been observed in `nsec3` system test: https://gitlab.isc.org/isc-projects/bind9/-/jobs/3223579. It seems to be an intermittent failure that isn't reliably reproducible.
```
I:nsec3:verify DNSSEC for zone nsec3-to-rsasha1-ds.kasp (110)
I:nsec3:error: DNSSEC verify failed for zone nsec3-to-rsasha1-ds.kasp
I:nsec3:failed
```
The dnssec-verify failed with:
```
Verifying the zone using the following algorithms:
- RSASHA1
- ECDSAP256SHA256
Missing ZSK for algorithm RSASHA1
Missing ZSK for algorithm ECDSAP256SHA256
No correct ECDSAP256SHA256 signature for a.nsec3-to-rsasha1-ds.kasp NSEC
No correct ECDSAP256SHA256 signature for b.nsec3-to-rsasha1-ds.kasp NSEC
No correct ECDSAP256SHA256 signature for c.nsec3-to-rsasha1-ds.kasp NSEC
The zone is not fully signed for the following algorithms:
RSASHA1
ECDSAP256SHA256
.
DNSSEC completeness test failed.
```
The signatures for the NSEC records weren't added for all algorithms. E.g. `a.nsec3-to-rsasha1-ds.kasp. NSEC`:
```
$ grep 'add re-sign.*a.nsec3-to-rsasha1-ds.kasp.*RRSIG.*NSEC ' ns3/named.run
09-Mar-2023 14:24:10.622 add re-sign a.nsec3-to-rsasha1-ds.kasp. 300 IN RRSIG NSEC 5 3 300 20230319032047 20230309132410 22352 nsec3-to-rsasha1-ds.kasp. PW2zoqereBVN2WjDe9+HdKMvupIP/QYxysrPfRmplfR+D/sxYklgWwI9 J6wCMyscR9xG8eKajn8c6z/9pxvVrGvMyP9eSmFWA/a0JWPw4a+yzttH wiMVBXqDA3E/V7q3K1P9NCTzVcR+XK+yrN/7JfraHQghTuE8beOCwsLR 2RohfIYb7VNlrnIo1B2mvIp0z7bPVeVqajy2V0z5SITK8HiJgbjZ5Jca lV6COMo2FVyxhNB9iI+RxI4r/JJdPU6OFBjK1GzvoUnDRVfz03izSBzg BIDNgpWhWyfe6P9vdWXZRUB8TPStHXW1W8CXubOXg97mtCTWKhloi9o+ ruMeBg==
```
Interestingly enough, the apex NSEC does have both signatures:
```
$ grep 'add re-sign.* nsec3-to-rsasha1-ds.kasp.*RRSIG.*NSEC ' ns3/named.run
09-Mar-2023 14:24:10.626 add re-sign nsec3-to-rsasha1-ds.kasp. 300 IN RRSIG NSEC 5 2 300 20230321155735 20230309132410 22352 nsec3-to-rsasha1-ds.kasp. ZScI7V66B0KmV5CkRIba/4D1l0ZVdG+XAocya2/4XY8nIYNAg8zU3EOH mOPsb4QwbmBKfc5qVJasAdpWeV4FedqN8yaF1iwtekzL82ual3Sm2GSy DfVNpFYbp1Mg+bERgUe3EFMSujmOpdF8m5YjIkWSN8kmVCuwRgc0DR34 8kQT9aJ27S8S4D/1b9MO5MSTvhkBoYShr894f4x5X5WJypqYi1xUdCeB X+3gtjqhn0/A/TnP+m83iQ1L9MoHQS0p1LovAsC6K9nWswp3IE55a11D ld3YGJt8g09uuS4BiofHFLlb7fSi1U9LQMiXVXhpztrjfQk7ma1OWv6u GY+gpg==
09-Mar-2023 14:24:10.626 add re-sign nsec3-to-rsasha1-ds.kasp. 300 IN RRSIG NSEC 13 2 300 20230321155735 20230309132410 47231 nsec3-to-rsasha1-ds.kasp. s9PtE0eacEjEML1dfL48KUBXGj8aISaetvPwPUcAD6U7+prNNWTTW536 8+QvIlM4t74aXWFe3PnsFH3QaGoi0g==
...
```
And for non-NSEC records, both signatures are there as well, e.g. `a.nsec3-to-rsasha1-ds.kasp. A`:
```
grep 'add re-sign.*a.nsec3-to-rsasha1-ds.kasp.*RRSIG.*A ' ns3/named.run
09-Mar-2023 14:24:05.222 add re-sign a.nsec3-to-rsasha1-ds.kasp. 300 IN RRSIG A 13 3 300 20230317145326 20230309132405 47231 nsec3-to-rsasha1-ds.kasp. 6TJmT7qJJIqQYg6ILbRWekcrlIZX67qHVq1jysMf2vopoCR9avLbaogq 2rtyMEQJ0JhtFg2taRI17EFgMjX4+A==
09-Mar-2023 14:24:10.622 add re-sign a.nsec3-to-rsasha1-ds.kasp. 300 IN RRSIG A 5 3 300 20230319032047 20230309132410 22352 nsec3-to-rsasha1-ds.kasp. hM0G1NY/V8rudj+suz86tOK+aTWkicF2KyipE9ao2dQFGX4F/IpUtLPd Akn6mnlC4G6PNjch1c1evWzPjynxnGuO8jv0QjC8sl/F9UHTANDq7o36 P1Orn/n9POcBs9OKk0GQNwfMdUrIzPxJuFfEsJzzNZi3fA/5GLvN9tBn vQqzx9+t99v64wk+O+Cno4uYAMMjz+wR6JKOWAqUJ6JT0hLQ+sNamXTU yRGBaCZf5gNbz5VZWUsuxv9FT/gWEc6X/mwEia8TkAwlP+VAvoiL8iDo Azma2qOUWiSM+n4/Lr9xl+9He9DHy6Cpjk5AVF4W4wTiES3gxwKuS06q 98CqwQ==
```April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)https://gitlab.isc.org/isc-projects/bind9/-/issues/3933Add GCC static analyzer to CI2023-07-18T06:09:17ZMichal NowakAdd GCC static analyzer to CIAdd Fedora CI job with GCC static analyzer (`-fanalyzer`).
Blocking issues:
- isc-projects/bind9#3929
- isc-projects/bind9#3930
- isc-projects/bind9#3931
- isc-projects/bind9#3932Add Fedora CI job with GCC static analyzer (`-fanalyzer`).
Blocking issues:
- isc-projects/bind9#3929
- isc-projects/bind9#3930
- isc-projects/bind9#3931
- isc-projects/bind9#3932Not plannedMichal NowakMichal Nowakhttps://gitlab.isc.org/isc-projects/bind9/-/issues/3925system test `serve-stale` doesn't get refreshed answer when expected2023-03-16T11:04:11ZTom Krizeksystem test `serve-stale` doesn't get refreshed answer when expectedJob [#3214712](https://gitlab.isc.org/isc-private/bind9/-/jobs/3214712) failed with:
```
I:serve-stale:check stale data.example TXT was refreshed (stale-answer-client-timeout 0 stale-refresh-time 4) (183)
I:serve-stale:failed
```
The t...Job [#3214712](https://gitlab.isc.org/isc-private/bind9/-/jobs/3214712) failed with:
```
I:serve-stale:check stale data.example TXT was refreshed (stale-answer-client-timeout 0 stale-refresh-time 4) (183)
I:serve-stale:failed
```
The test was expecting a fresh answer, but it got a stale one instead:
```
; <<>> DiG 9.18.13-S1 <<>> +time +tries -p 14092 @10.53.0.3 data.example TXT
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62349
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 8c969d3b5a853b08010000006405d40e4781607af7f2ae49 (good)
; EDE: 3 (Stale Answer): (stale data prioritized over lookup)
;; QUESTION SECTION:
;data.example. IN TXT
;; ANSWER SECTION:
data.example. 3 IN TXT "A text record with a 2 second ttl"
;; Query time: 23 msec
;; SERVER: 10.53.0.3#14092(10.53.0.3) (UDP)
;; WHEN: Mon Mar 06 11:52:46 UTC 2023
;; MSG SIZE rcvd: 155
```
Saw this in ~"v9.18", but the exact same test is present in other branches, so I assume they're affected as well.April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/3924missing messages in `dnstap` system test after query failure2023-03-13T12:01:09ZTom Krizekmissing messages in `dnstap` system test after query failureIn job [#3215476](https://gitlab.isc.org/isc-private/bind9/-/jobs/3215476), the tests expects two UDP messages in the dnstap output - `CLIENT_QUERY` and `CLIENT_RESPONSE` after querying for `a.example`. The [`dnstap.out`](https://gitlab....In job [#3215476](https://gitlab.isc.org/isc-private/bind9/-/jobs/3215476), the tests expects two UDP messages in the dnstap output - `CLIENT_QUERY` and `CLIENT_RESPONSE` after querying for `a.example`. The [`dnstap.out`](https://gitlab.isc.org/isc-private/bind9/-/jobs/3215476/artifacts/file/bin/tests/system/dnstap/dnstap.out) doesn't contain any messages.
```
I:dnstap:checking reopened unix socket message counts
I:dnstap:checking UDP message counts
I:dnstap:ns4 0 expected 2
I:dnstap:failed
I:dnstap:checking TCP message counts
I:dnstap:checking AUTH_QUERY message counts
I:dnstap:checking AUTH_RESPONSE message counts
I:dnstap:checking CLIENT_QUERY message counts
I:dnstap:ns4 0 expected 1
I:dnstap:failed
I:dnstap:checking CLIENT_RESPONSE message counts
I:dnstap:ns4 0 expected 1
I:dnstap:failed
```
Inspecting the [`ns4/named.run`](https://gitlab.isc.org/isc-private/bind9/-/jobs/3215476/artifacts/file/bin/tests/system/dnstap/ns4/named.run), it looks like reopening the dnstap socket happened as expected. Afterwards, a query for a.example was received. For some reason, the query failed to resolve. It looks like before the server had a chance to respond to the query, it was shut down. The [`dig.out`](https://gitlab.isc.org/isc-private/bind9/-/jobs/3215476/artifacts/file/bin/tests/system/dnstap/dnstap.out) is also empty.
```
06-Mar-2023 14:48:09.490 received control channel command 'dnstap -reopen'
06-Mar-2023 14:48:09.490 reopening dnstap destination 'dnstap.out'
06-Mar-2023 14:48:09.550 client @0x7f1bed89ff68 10.53.0.1#59720: UDP request
06-Mar-2023 14:48:09.550 query client=0x7f1bed89ff68 thread=0x7f1c197ff700(a.example/A): client attr:0x2302, query attr:0x703, restarts:0, origqname:a.example, timer:0, authdb:0, referral:0
06-Mar-2023 14:48:09.550 query client=0x7f1bed89ff68 thread=0x7f1c197ff700(a.example/A): query_gotanswer: unexpected error: failure
06-Mar-2023 14:48:09.554 client @0x7f1bed89ff68 10.53.0.1#59720 (a.example): query failed (failure) for a.example/IN/A at query.c:7779
06-Mar-2023 14:48:16.306 shutting down
06-Mar-2023 14:48:16.310 client @0x7f1bed89ff68 10.53.0.1#59720: freeing client
06-Mar-2023 14:48:16.310 query client=0x7f1bed89ff68 thread=0x7f1c20fff700(<unknown-query>): query_reset
06-Mar-2023 14:48:16.322 closing dnstap
```April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/3923timing issue with incoming XoT check in `doth` system test2023-03-09T10:42:44ZTom Krizektiming issue with incoming XoT check in `doth` system testIn job [#3214134](https://gitlab.isc.org/isc-private/bind9/-/jobs/3214134):
```
I:doth:testing incoming XoT functionality (from the first secondary, StrictTLS via specified IPv6) (5)
I:doth:failed
```
The check failed because the file ...In job [#3214134](https://gitlab.isc.org/isc-private/bind9/-/jobs/3214134):
```
I:doth:testing incoming XoT functionality (from the first secondary, StrictTLS via specified IPv6) (5)
I:doth:failed
```
The check failed because the file `ns2/example6.db` didn't exist at the time when it's checked (which happens right after client dig AXFR succeeds). The file is present in the artifacts. From the log, it seems that the file was written shortly after the client AXFR has completed.
```
06-Mar-2023 09:32:09.973 zone example6/IN: Transfer started.
06-Mar-2023 09:32:10.301 zone example6/IN: zone transfer finished: success
06-Mar-2023 09:32:10.301 zone_dump: zone example6/IN: enter
06-Mar-2023 09:32:11.789 client @0x7fe9ab435d68 10.53.0.10#44113 (example6): AXFR request
06-Mar-2023 09:32:11.801 client @0x7fe9ab435d68 10.53.0.10#44113 (example6): transfer of 'example6/IN': AXFR ended: 5 messages, 2676 records, 55815 bytes, 0.011 secs (5074090 bytes/sec) (serial 1397051952)
06-Mar-2023 09:32:12.409 zone_gotwritehandle: zone example6/IN: enter
06-Mar-2023 09:32:12.421 dump_done: zone example6/IN: enter
06-Mar-2023 09:32:12.421 zone_journal_compact: zone example6/IN: target journal size 53044
```
Possible fix: Add retry to the file check.April 2023 (9.16.40, 9.16.40-S1, 9.18.14, 9.18.14-S1, 9.19.12)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/3917Named should log UV version when starting up2023-04-19T21:26:25ZMark AndrewsNamed should log UV version when starting upWe already report UV version via rndc. This should have been added at the same time.We already report UV version via rndc. This should have been added at the same time.March 2023 (9.16.39, 9.16.39-S1, 9.18.13, 9.18.13-S1, 9.19.11)