BIND issueshttps://gitlab.isc.org/isc-projects/bind9/-/issues2020-10-26T12:30:11Zhttps://gitlab.isc.org/isc-projects/bind9/-/issues/21679.11.23: undefined reference to `isc_atomic_xadd' on arm2020-10-26T12:30:11ZPetr Menšík9.11.23: undefined reference to `isc_atomic_xadd' on arm<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please do *NOT* report it here, but send an
email to [...<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please do *NOT* report it here, but send an
email to [security-officer@isc.org](security-officer@isc.org).
-->
### Summary
undefined reference to `isc_atomic_xadd' on 9.11.23
### BIND version used
9.11.23
### Steps to reproduce
build bind on arch without atomic support in lib/isc:
- armv7hl
- aarch64
- s390x
### What is the current *bug* behavior?
I made new test build on Fedora. [BIND 9.11.23 fails](https://koji.fedoraproject.org/koji/taskinfo?taskID=51689295), where the same package with [previous release passes](https://koji.fedoraproject.org/koji/taskinfo?taskID=51689267).
### What is the expected *correct* behavior?
Even platforms without explicit support for atomic operations still work.
### Relevant configuration files
```
../configure --build=armv7hl-redhat-linux-gnu --host=armv7hl-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-python=/usr/bin/python3 --with-libtool --localstatedir=/var --enable-threads --enable-ipv6 --enable-filter-aaaa --with-pic --disable-static --includedir=/usr/include/bind9 --with-tuning=large --with-libidn2 --enable-openssl-hash --with-geoip2 --enable-native-pkcs11 --with-pkcs11=/usr/lib/pkcs11/libsofthsm2.so --with-dlopen=yes --with-dlz-ldap=yes --with-dlz-postgres=yes --with-dlz-mysql=yes --with-dlz-filesystem=yes --with-gssapi=yes --disable-isc-spnego --with-lmdb=yes --with-libjson --enable-dnstap --with-cmocka --enable-fixed-rrset --with-docbook-xsl=/usr/share/sgml/docbook/xsl-stylesheets --enable-full-report
```
### Relevant logs and/or screenshots
```
libtool: compile: gcc -I/builddir/build/BUILD/bind-9.11.23/build -I../../../.. -I. -Iinclude -I/builddir/build/BUILD/bind-9.11.23/build/lib/dns/include -I../../../../lib/dns/include -I/builddir/build/BUILD/bind-9.11.23/build/lib/isc/include -I../../../../lib/isc -I../../../../lib/isc/include -I../../../../lib/isc/unix/include -I../../../../lib/isc/pthreads/include -I../../../../lib/isc/noatomic/include -D_REENTRANT -DOPENSSL -DTESTS=\"/builddir/build/BUILD/bind-9.11.23/build/lib/dns/tests/\" -DDIG_SIGCHASE -D_GNU_SOURCE -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -DPK11_FLAVOR=PK11_SOFTHSMV2_FLAVOR -I/usr/include/libxml2 -I/usr/include/google -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -fno-strict-aliasing -fno-delete-null-pointer-checks -c ../../../../lib/dns/tests/zonemgr_test.c -fPIC -DPIC -o .libs/zonemgr_test.o
/bin/sh /builddir/build/BUILD/bind-9.11.23/build/libtool --mode=link --tag=CC gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -DPK11_FLAVOR=PK11_SOFTHSMV2_FLAVOR -I/usr/include/libxml2 -I/usr/include/google -fPIC \
-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--export-dynamic -o db_test db_test.lo dnstest.lo ../libdns.la -lmaxminddb -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lcrypto \
../../isc/libisc.la -ldl -lprotobuf-c -lfstrm -lcap -lz -ljson-c -llmdb -lpthread -lxml2 -lcmocka
/bin/sh /builddir/build/BUILD/bind-9.11.23/build/libtool --mode=link --tag=CC gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -DPK11_FLAVOR=PK11_SOFTHSMV2_FLAVOR -I/usr/include/libxml2 -I/usr/include/google -fPIC \
-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--export-dynamic -o dbdiff_test dbdiff_test.lo dnstest.lo \
../libdns.la -lmaxminddb -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lcrypto ../../isc/libisc.la -ldl -lprotobuf-c -lfstrm -lcap -lz -ljson-c -llmdb -lpthread -lxml2 -lcmocka
libtool: compile: gcc -I/builddir/build/BUILD/bind-9.11.23/build -I../../../.. -I. -Iinclude -I/builddir/build/BUILD/bind-9.11.23/build/lib/dns/include -I../../../../lib/dns/include -I/builddir/build/BUILD/bind-9.11.23/build/lib/isc/include -I../../../../lib/isc -I../../../../lib/isc/include -I../../../../lib/isc/unix/include -I../../../../lib/isc/pthreads/include -I../../../../lib/isc/noatomic/include -D_REENTRANT -DOPENSSL -DTESTS=\"/builddir/build/BUILD/bind-9.11.23/build/lib/dns/tests/\" -DDIG_SIGCHASE -D_GNU_SOURCE -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -DPK11_FLAVOR=PK11_SOFTHSMV2_FLAVOR -I/usr/include/libxml2 -I/usr/include/google -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -fno-strict-aliasing -fno-delete-null-pointer-checks -c ../../../../lib/dns/tests/zt_test.c -fPIC -DPIC -o .libs/zt_test.o
libtool: link: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -DPK11_FLAVOR=PK11_SOFTHSMV2_FLAVOR -I/usr/include/libxml2 -I/usr/include/google -fPIC -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--export-dynamic -o .libs/acl_test .libs/acl_test.o .libs/dnstest.o ../.libs/libdns.so /builddir/build/BUILD/bind-9.11.23/build/lib/isc/.libs/libisc.so -lmaxminddb -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err ../../isc/.libs/libisc.so -lcrypto -ldl -lprotobuf-c -lfstrm -lcap -lz -ljson-c -llmdb -lpthread -lxml2 -lcmocka
/bin/sh /builddir/build/BUILD/bind-9.11.23/build/libtool --mode=link --tag=CC gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -DPK11_FLAVOR=PK11_SOFTHSMV2_FLAVOR -I/usr/include/libxml2 -I/usr/include/google -fPIC \
-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--export-dynamic -o dbiterator_test dbiterator_test.lo dnstest.lo \
../libdns.la -lmaxminddb -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lcrypto ../../isc/libisc.la -ldl -lprotobuf-c -lfstrm -lcap -lz -ljson-c -llmdb -lpthread -lxml2 -lcmocka
libtool: link: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -DPK11_FLAVOR=PK11_SOFTHSMV2_FLAVOR -I/usr/include/libxml2 -I/usr/include/google -fPIC -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--export-dynamic -o .libs/db_test .libs/db_test.o .libs/dnstest.o ../.libs/libdns.so /builddir/build/BUILD/bind-9.11.23/build/lib/isc/.libs/libisc.so -lmaxminddb -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err ../../isc/.libs/libisc.so -lcrypto -ldl -lprotobuf-c -lfstrm -lcap -lz -ljson-c -llmdb -lpthread -lxml2 -lcmocka
libtool: link: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -DPK11_FLAVOR=PK11_SOFTHSMV2_FLAVOR -I/usr/include/libxml2 -I/usr/include/google -fPIC -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--export-dynamic -o .libs/dbdiff_test .libs/dbdiff_test.o .libs/dnstest.o ../.libs/libdns.so /builddir/build/BUILD/bind-9.11.23/build/lib/isc/.libs/libisc.so -lmaxminddb -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err ../../isc/.libs/libisc.so -lcrypto -ldl -lprotobuf-c -lfstrm -lcap -lz -ljson-c -llmdb -lpthread -lxml2 -lcmocka
/usr/bin/ld: ../.libs/libdns.so: undefined reference to `isc_atomic_xadd'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:519: acl_test] Error 1
make[3]: *** Waiting for unfinished jobs....
/usr/bin/ld: ../.libs/libdns.so: undefined reference to `isc_atomic_xadd'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:524: db_test] Error 1
/usr/bin/ld: ../.libs/libdns.so: undefined reference to `isc_atomic_xadd'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:529: dbdiff_test] Error 1
libtool: link: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -DPK11_FLAVOR=PK11_SOFTHSMV2_FLAVOR -I/usr/include/libxml2 -I/usr/include/google -fPIC -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--export-dynamic -o .libs/dbiterator_test .libs/dbiterator_test.o .libs/dnstest.o ../.libs/libdns.so /builddir/build/BUILD/bind-9.11.23/build/lib/isc/.libs/libisc.so -lmaxminddb -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err ../../isc/.libs/libisc.so -lcrypto -ldl -lprotobuf-c -lfstrm -lcap -lz -ljson-c -llmdb -lpthread -lxml2 -lcmocka
/usr/bin/ld: ../.libs/libdns.so: undefined reference to `isc_atomic_xadd'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:534: dbiterator_test] Error 1
make[3]: Leaving directory '/builddir/build/BUILD/bind-9.11.23/build/lib/dns/tests'
make[2]: *** [Makefile:292: testdirs] Error 1
make[2]: Leaving directory '/builddir/build/BUILD/bind-9.11.23/build/lib/dns'
make[1]: *** [Makefile:82: subdirs] Error 1
make[1]: Leaving directory '/builddir/build/BUILD/bind-9.11.23/build/lib'
make: *** [Makefile:88: subdirs] Error 1
error: Bad exit status from /var/tmp/rpm-tmp.dP3EtO (%build)
Bad exit status from /var/tmp/rpm-tmp.dP3EtO (%build)
```
### Possible fixes
Have not yet found cause, or fix. No locking change was mentioned in release notes.https://gitlab.isc.org/isc-projects/bind9/-/issues/2164ThreadSanitizer: data race bin/named/controlconf.c:257:22 in control_senddone2020-09-24T11:34:52ZOndřej SurýThreadSanitizer: data race bin/named/controlconf.c:257:22 in control_senddoneWhile this looks innocuous (it's a double write to `listener->listening = true` from two different threads), it also might be a manifestation of something more serious:
* [ ] [c4ae392dcfa8039dec6ea9d8b889585cf1b4dac39600c33ff60ec627014e...While this looks innocuous (it's a double write to `listener->listening = true` from two different threads), it also might be a manifestation of something more serious:
* [ ] [c4ae392dcfa8039dec6ea9d8b889585cf1b4dac39600c33ff60ec627014e6b8e.txt](/uploads/f575d02bb3bd86024666754d8fb9eea2/c4ae392dcfa8039dec6ea9d8b889585cf1b4dac39600c33ff60ec627014e6b8e.txt)October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.16.8-S1, 9.17.6)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/2160ThreadSanitizer: data race bin/named/controlconf.c:422:37 in control_recvmessage2020-09-24T11:31:45ZOndřej SurýThreadSanitizer: data race bin/named/controlconf.c:422:37 in control_recvmessage* [x] [15696694a1fb8757cb48d39bf0c93bbc5593080f7056b7316abda2cc91c40c19.txt](/uploads/5a7601e4a5ecec9fe4fd28fca90f79d1/15696694a1fb8757cb48d39bf0c93bbc5593080f7056b7316abda2cc91c40c19.txt)* [x] [15696694a1fb8757cb48d39bf0c93bbc5593080f7056b7316abda2cc91c40c19.txt](/uploads/5a7601e4a5ecec9fe4fd28fca90f79d1/15696694a1fb8757cb48d39bf0c93bbc5593080f7056b7316abda2cc91c40c19.txt)October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.16.8-S1, 9.17.6)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/2155reference leak on error in controlconf.c:control_respond2020-09-24T11:26:02ZMark Andrewsreference leak on error in controlconf.c:control_respondon inspection this should be fixed.
```
diff --git a/bin/named/controlconf.c b/bin/named/controlconf.c
index 5d776e638e..706ec0b9ec 100644
--- a/bin/named/controlconf.c
+++ b/bin/named/controlconf.c
@@ -371,8 +371,10 @@ control_respond(...on inspection this should be fixed.
```
diff --git a/bin/named/controlconf.c b/bin/named/controlconf.c
index 5d776e638e..706ec0b9ec 100644
--- a/bin/named/controlconf.c
+++ b/bin/named/controlconf.c
@@ -371,8 +371,10 @@ control_respond(isc_nmhandle_t *handle, isc_result_t result, void *arg) {
}
return;
+
cleanup:
conn_cleanup(conn);
+ isc_nmhandle_detach(&conn->cmdhandle);
}
static void
```October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.16.8-S1, 9.17.6)https://gitlab.isc.org/isc-projects/bind9/-/issues/2154Investigate rpz system test failure2023-10-23T13:59:27ZMark AndrewsInvestigate rpz system test failureJob [#1161531](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1161531) failed for c5c2a4820b6dd705443e42a515cd20fc4293d35b:Job [#1161531](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1161531) failed for c5c2a4820b6dd705443e42a515cd20fc4293d35b:BIND 9.19.xhttps://gitlab.isc.org/isc-projects/bind9/-/issues/2153Rebuild RBTDB while rehashing2021-10-05T15:35:42ZBrian ConryRebuild RBTDB while rehashing@ondrej had an idea related to rebuilding the RBTDB while rehashing as a means of clearing out empty interior nodes.
This issue is a reminder.
Description to be updated and amended.@ondrej had an idea related to rebuilding the RBTDB while rehashing as a means of clearing out empty interior nodes.
This issue is a reminder.
Description to be updated and amended.BIND 9.19.xOndřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/2152Bind 9.17.4 assertion failure2020-09-16T10:25:41ZKlaus DarilionBind 9.17.4 assertion failure### Summary
After upgrade from Bind 9.11.22 to 9.17.4 Bind crashes on startup with an assertion failure.
### BIND version used
BIND 9.17.4-1+ubuntu18.04.1+isc+5-Ubuntu (Development Release) <id:>
running on Linux x86_64 4.15.0-112-gen...### Summary
After upgrade from Bind 9.11.22 to 9.17.4 Bind crashes on startup with an assertion failure.
### BIND version used
BIND 9.17.4-1+ubuntu18.04.1+isc+5-Ubuntu (Development Release) <id:>
running on Linux x86_64 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020
built by make with '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/x86_64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--disable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=yes' '--with-libidn2' '--with-libjson-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' '--enable-dnstap' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fdebug-prefix-map=/build/bind9-nHvHYZ/bind9-9.17.4=. -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
compiled by GCC 7.5.0
compiled with OpenSSL version: OpenSSL 1.1.1 11 Sep 2018
linked to OpenSSL version: OpenSSL 1.1.1 11 Sep 2018
compiled with libuv version: 1.38.1
linked to libuv version: 1.38.1
compiled with libxml2 version: 2.9.4
linked to libxml2 version: 20904
compiled with json-c version: 0.12.1
linked to json-c version: 0.12.1
compiled with zlib version: 1.2.11
linked to zlib version: 1.2.11
linked to maxminddb version: 1.3.2
compiled with protobuf-c version: 1.3.1
linked to protobuf-c version: 1.3.1
threads support is enabled
default paths:
named configuration: /etc/bind/named.conf
rndc configuration: /etc/bind/rndc.conf
DNSSEC root key: /etc/bind/bind.keys
nsupdate session key: //run/named/session.key
named PID file: //run/named/named.pid
named lock file: //run/named/named.lock
geoip-directory: /usr/share/GeoIP
### Steps to reproduce
I do not know. Just startin up the server in our cases generates the assertion failure.
### What is the current *bug* behavior?
Assertion failure
### What is the expected *correct* behavior?
Startup
### Relevant configuration files
I can't share the whole config. Nothing special in it. Here some options:
```
options {
directory "/etc/bind-validation/zones";
auth-nxdomain no; # conform to RFC1035
listen-on { xx.xxx.34.22; };
listen-on-v6 { xxx:xxx:9::7; };
version "IPCom ancast network 1.0";
notify-source xx.xxx.34.22;
notify-source-v6 xxx:xxx:9::7;
transfer-source xx.xxx.34.22;
transfer-source-v6 xxx:xxx:9::7;
try-tcp-refresh no;
notify explicit;
query-source xx.xxx.34.22;
query-source-v6 xxx:xxx:9::7;
/* Disable all zone checks. This should speed up zone loading */
check-names master ignore;
check-names slave ignore;
check-dup-records ignore;
check-mx ignore;
check-wildcard no;
check-integrity no;
check-mx-cname ignore;
check-srv-cname ignore;
check-sibling no;
check-spf ignore;
transfers-in 50; // number of total concurrent zone transfers from the masters to me
transfers-out 200; // number of concurrent zone transfers from me to my slaves
transfers-per-ns 25; // number of concurrent zone transfers per master from the masters to me
max-refresh-time 300;
max-retry-time 300;
// some customers do not allow IXFR. Hence we have to calculate the "diff" ourselfs. Otherwise all
// the zone transfers to the anycast nodes would be AXFR always. (.foo full zone to Singapore
// takes up 1 one hour)
ixfr-from-differences yes; // default: no
request-ixfr yes; // default: yes
max-journal-size 50m; // default: unlimited
pid-file "/var/run/named/named-validation.pid";
allow-recursion {
none;
};
allow-transfer {
...
};
allow-query {
...
};
};
controls {
...
};
logging {
channel default_syslog {
syslog local1;
};
};
```
### Relevant logs and/or screenshots
```
09:05:43 named[5112]: zone asdfasdf.asdfsdf/IN: sending notifies (serial 1600236090)
09:05:43 named[5112]: zone asdfasdfas.dasfdfasdf/IN: sending notifies (serial 1600236087)
09:05:43 named[5112]: zone asfdfasdf.asdfasdfsadf/IN: sending notifies (serial 1600236077)
09:05:43 named[5112]: netmgr/netmgr.c:1112: REQUIRE(((__builtin_expect(!!((handle) != ((void *)0)), 1) && __builtin_expect(!!(((const isc__magic_t *)(handle))->magic == ((('N') << 24 | ('M') << 16 | ('H') << 8 | ('D')))), 1)) && __extension__ ({ __auto_type __atomic_load_ptr = (&(handle)->references); __typeof__ (*__atomic_load_ptr) __atomic_load_tmp; __atomic_load (__atomic_load_ptr, &__atomic_load_tmp, (5)); __atomic_load_tmp; }) > 0)) failed, back trace
09:05:43 named[5112]: zone asdfasdf.asdfasdf/IN: sending notifies (serial 1600236059)
09:05:43 named[5112]: zone sadfsdfasdf.asdfasdf/IN: sending notifies (serial 2020091603)
09:05:43 named[5112]: /usr/sbin/named(+0x2b4cb) [0x5639af3984cb]
09:05:43 named[5112]: zone sdfasdfasdf/IN: sending notifies (serial 1512401053)
09:05:43 named[5112]: zone asfasdf.asdfsdf/IN: sending notifies (serial 1600236082)
09:05:43 named[5112]: /usr/lib/x86_64-linux-gnu/libisc.so.1704(isc_assertion_failed+0xa) [0x7f39dfffe16a]
09:05:43 named[5112]: /usr/lib/x86_64-linux-gnu/libisc.so.1704(isc_nmhandle_ref+0x5a) [0x7f39dffe32ba]
09:05:43 named[5112]: /usr/sbin/named-(+0x28d5b) [0x5639af395d5b]
09:05:43 named[5112]: /usr/lib/x86_64-linux-gnu/libisc.so.1704(+0x4dfef) [0x7f39e001dfef]
09:05:43 named[5112]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f39ddfe16db]
09:05:43 named[5112]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f39dd521a3f]
09:05:43 named[5112]: exiting (due to assertion failure)
```
### Possible fixes
Nonehttps://gitlab.isc.org/isc-projects/bind9/-/issues/2151Use after free in named2020-09-24T11:25:32ZMark AndrewsUse after free in namedJob [#1157683](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1157683) failed for fe72a28e1ba3488be9af1095c27a756d2dbae921:
```
D:shutdown:=================================================================
7574D:shutdown:==26090==ERROR...Job [#1157683](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1157683) failed for fe72a28e1ba3488be9af1095c27a756d2dbae921:
```
D:shutdown:=================================================================
7574D:shutdown:==26090==ERROR: AddressSanitizer: heap-use-after-free on address 0x616000110688 at pc 0x55d9c564a410 bp 0x7f7b91acdb80 sp 0x7f7b91acdb78
7575D:shutdown:READ of size 8 at 0x616000110688 thread T14
7576D:shutdown:#0 0x55d9c564a40f in conn_cleanup /builds/isc-projects/bind9/bin/named/controlconf.c:285
7577D:shutdown:#1 0x55d9c5653b32 in control_command /builds/isc-projects/bind9/bin/named/controlconf.c:387
7578D:shutdown:#2 0x7f7ba1ede0c9 in dispatch /builds/isc-projects/bind9/lib/isc/task.c:1152
7579D:shutdown:#3 0x7f7ba1ede0c9 in run /builds/isc-projects/bind9/lib/isc/task.c:1344
7580D:shutdown:#4 0x7f7b9f408fa2 in start_thread /build/glibc-vjB4T1/glibc-2.28/nptl/pthread_create.c:486
7581D:shutdown:#5 0x7f7b9e4224ce in clone (/lib/x86_64-linux-gnu/libc.so.6+0xf94ce)
7582D:shutdown:
7583D:shutdown:0x616000110688 is located 520 bytes inside of 569-byte region [0x616000110480,0x6160001106b9)
7584D:shutdown:freed by thread T8 here:
7585D:shutdown:#0 0x7f7ba22a8fb0 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe8fb0)
7586D:shutdown:#1 0x7f7ba1e8cb67 in default_memfree /builds/isc-projects/bind9/lib/isc/mem.c:742
7587D:shutdown:#2 0x7f7ba1eada0d in mem_put /builds/isc-projects/bind9/lib/isc/mem.c:654
7588D:shutdown:#3 0x7f7ba1eada0d in isc___mem_put /builds/isc-projects/bind9/lib/isc/mem.c:1110
7589D:shutdown:#4 0x7f7ba1ea4d09 in isc__mem_put /builds/isc-projects/bind9/lib/isc/mem.c:2439
7590D:shutdown:#5 0x7f7ba1dbfabc in nmhandle_free netmgr/netmgr.c:1190
7591D:shutdown:#6 0x7f7ba1dcb2f6 in nmsocket_cleanup netmgr/netmgr.c:773
7592D:shutdown:#7 0x7f7ba1dcce04 in nmsocket_maybe_destroy netmgr/netmgr.c:866
7593D:shutdown:#8 0x7f7ba1dcd2d2 in isc__nmsocket_prep_destroy netmgr/netmgr.c:912
7594D:shutdown:#9 0x7f7ba1dd21b3 in tcp_close_cb netmgr/tcp.c:1082
7595D:shutdown:#10 0x7f7b9f43d044 in uv_run (/usr/lib/x86_64-linux-gnu/libuv.so.1+0x11044)
7596D:shutdown:#11 0x7f7b94fdfc9f (<unknown module>)
7597D:shutdown:
7598D:shutdown:previously allocated by thread T8 here:
7599D:shutdown:#0 0x7f7ba22a9330 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9330)
7600D:shutdown:#1 0x7f7ba1e8cc24 in default_memalloc /builds/isc-projects/bind9/lib/isc/mem.c:713
7601D:shutdown:#2 0x7f7ba1ea9c56 in mem_get /builds/isc-projects/bind9/lib/isc/mem.c:622
7602D:shutdown:#3 0x7f7ba1ea9c56 in isc___mem_get /builds/isc-projects/bind9/lib/isc/mem.c:1044
7603D:shutdown:#4 0x7f7ba1ea4054 in isc__mem_get /builds/isc-projects/bind9/lib/isc/mem.c:2432
7604D:shutdown:#5 0x7f7ba1dc4eb3 in alloc_handle netmgr/netmgr.c:1063
7605D:shutdown:#6 0x7f7ba1dc4eb3 in isc__nmhandle_get netmgr/netmgr.c:1087
7606D:shutdown:#7 0x7f7ba1dd9758 in isc__nm_async_tcpchildaccept netmgr/tcp.c:491
7607D:shutdown:#8 0x7f7ba1dcf8d4 in process_queue netmgr/netmgr.c:628
7608D:shutdown:#9 0x7f7ba1dd087f in async_cb netmgr/netmgr.c:596
7609D:shutdown:#10 0x7f7b9f43c667 (/usr/lib/x86_64-linux-gnu/libuv.so.1+0x10667)
7610D:shutdown:
7611D:shutdown:Thread T14 created by T0 here:
7612D:shutdown:#0 0x7f7ba2210db0 in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x50db0)
7613D:shutdown:#1 0x7f7ba1f05c5a in isc_thread_create pthreads/thread.c:73
7614D:shutdown:#2 0x7f7ba1ee6a28 in isc_taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1434
7615D:shutdown:#3 0x55d9c5661339 in create_managers /builds/isc-projects/bind9/bin/named/main.c:915
7616D:shutdown:#4 0x55d9c5661339 in setup /builds/isc-projects/bind9/bin/named/main.c:1223
7617D:shutdown:#5 0x55d9c5661339 in main /builds/isc-projects/bind9/bin/named/main.c:1523
7618D:shutdown:#6 0x7f7b9e34d09a in __libc_start_main ../csu/libc-start.c:308
7619D:shutdown:
7620D:shutdown:Thread T8 created by T0 here:
7621D:shutdown:#0 0x7f7ba2210db0 in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x50db0)
7622D:shutdown:#1 0x7f7ba1f05c5a in isc_thread_create pthreads/thread.c:73
7623D:shutdown:#2 0x7f7ba1dc1176 in isc_nm_start netmgr/netmgr.c:223
7624D:shutdown:#3 0x55d9c5661304 in create_managers /builds/isc-projects/bind9/bin/named/main.c:909
7625D:shutdown:#4 0x55d9c5661304 in setup /builds/isc-projects/bind9/bin/named/main.c:1223
7626D:shutdown:#5 0x55d9c5661304 in main /builds/isc-projects/bind9/bin/named/main.c:1523
7627D:shutdown:#6 0x7f7b9e34d09a in __libc_start_main ../csu/libc-start.c:308
7628D:shutdown:
7629D:shutdown:SUMMARY: AddressSanitizer: heap-use-after-free /builds/isc-projects/bind9/bin/named/controlconf.c:285 in conn_cleanup
```
```
I:shutdown:stopping servers
7705I:shutdown:Core dump(s) found: shutdown/resolver/core.26090
7706D:shutdown:backtrace from shutdown/resolver/core.26090:
7707D:shutdown:--------------------------------------------------------------------------------
7708D:shutdown:Core was generated by `/builds/isc-projects/bind9/bin/named/.libs/named -c /builds/isc-projects/bind9/'.
7709D:shutdown:Program terminated with signal SIGABRT, Aborted.
7710D:shutdown:#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
7711D:shutdown:[Current thread is 1 (Thread 0x7f7b91ace700 (LWP 26117))]
7712D:shutdown:#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
7713D:shutdown:#1 0x00007f7b9e34b535 in __GI_abort () at abort.c:79
7714D:shutdown:#2 0x00007f7ba22c6e6b in ?? () from /usr/lib/x86_64-linux-gnu/libasan.so.5
7715D:shutdown:#3 0x00007f7ba22ceed8 in ?? () from /usr/lib/x86_64-linux-gnu/libasan.so.5
7716D:shutdown:#4 0x00007f7ba22b397d in ?? () from /usr/lib/x86_64-linux-gnu/libasan.so.5
7717D:shutdown:#5 0x00007f7ba22b4308 in __asan_report_load8 () from /usr/lib/x86_64-linux-gnu/libasan.so.5
7718D:shutdown:#6 0x000055d9c564a410 in conn_cleanup (conn=conn@entry=0x6160001105e8) at controlconf.c:290
7719D:shutdown:#7 0x000055d9c5653b33 in control_command (task=<optimized out>, event=<optimized out>) at controlconf.c:387
7720D:shutdown:#8 0x00007f7ba1ede0ca in dispatch (threadid=<optimized out>, manager=<optimized out>) at task.c:1152
7721D:shutdown:#9 run (queuep=<optimized out>) at task.c:1344
7722D:shutdown:#10 0x00007f7b9f408fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
7723D:shutdown:#11 0x00007f7b9e4224cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
7724D:shutdown:--------------------------------------------------------------------------------
7725D:shutdown:full backtrace from shutdown/resolver/core.26090 saved in core.26090-backtrace.txt
7726D:shutdown:core dump shutdown/resolver/core.26090 archived as shutdown/resolver/core.26090.gz
```October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.16.8-S1, 9.17.6)https://gitlab.isc.org/isc-projects/bind9/-/issues/2150INSIST(ISC_LIST_EMPTY(res->dbuckets[i].list)); assertion failure during shutdown2020-12-16T22:13:12ZMark AndrewsINSIST(ISC_LIST_EMPTY(res->dbuckets[i].list)); assertion failure during shutdownJob [#1157253](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1157253) failed for 7e39ead11bfa4b109897e75683af5b03f5f89eb4:
```
I:rpzrecurse:Core dump(s) found: rpzrecurse/ns3/core.7743
6956D:rpzrecurse:backtrace from rpzrecurse/ns3/...Job [#1157253](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1157253) failed for 7e39ead11bfa4b109897e75683af5b03f5f89eb4:
```
I:rpzrecurse:Core dump(s) found: rpzrecurse/ns3/core.7743
6956D:rpzrecurse:backtrace from rpzrecurse/ns3/core.7743:
6957D:rpzrecurse:--------------------------------------------------------------------------------
6958D:rpzrecurse:Core was generated by `/builds/isc-projects/bind9/bin/named/.libs/lt-named -D rpzrecurse-ns3 -X named.'.
6959D:rpzrecurse:Program terminated with signal SIGABRT, Aborted.
6960D:rpzrecurse:#0 0x00007f7160dcb438 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
6961D:rpzrecurse:[Current thread is 1 (Thread 0x7f7151321700 (LWP 7793))]
6962D:rpzrecurse:#0 0x00007f7160dcb438 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
6963D:rpzrecurse:#1 0x00007f7160dcd03a in __GI_abort () at abort.c:89
6964D:rpzrecurse:#2 0x0000000000428ba5 in assertion_failed (file=<optimized out>, line=<optimized out>, type=isc_assertiontype_insist, cond=0x7f7163056920 "((res->dbuckets[i].list).head == ((void *)0))") at main.c:254
6965D:rpzrecurse:#3 0x00007f71632c689a in isc_assertion_failed (file=file@entry=0x7f71630536e0 "resolver.c", line=line@entry=10034, type=type@entry=isc_assertiontype_insist, cond=cond@entry=0x7f7163056920 "((res->dbuckets[i].list).head == ((void *)0))") at assertions.c:46
6966D:rpzrecurse:#4 0x00007f7162faadb7 in destroy (res=0x7f7163580340) at resolver.c:10034
6967D:rpzrecurse:#5 dns_resolver_detach (resp=resp@entry=0x7f7144005488) at resolver.c:10578
6968D:rpzrecurse:#6 0x00007f7162fea2cc in destroy (view=0x7f7144005460) at view.c:407
6969D:rpzrecurse:#7 0x00007f7162feb028 in dns_view_weakdetach (viewp=viewp@entry=0x7f7144006bd0) at view.c:727
6970D:rpzrecurse:#8 0x00007f7162ff7727 in zone_free (zone=0x7f7144006260) at zone.c:1197
6971D:rpzrecurse:#9 0x00007f716300d5cc in zone_shutdown (task=<optimized out>, event=<optimized out>) at zone.c:14069
6972D:rpzrecurse:#10 0x00007f71632e7629 in dispatch (threadid=<optimized out>, manager=<optimized out>) at task.c:1152
6973D:rpzrecurse:#11 run (queuep=<optimized out>) at task.c:1344
6974D:rpzrecurse:#12 0x00007f71619476ba in start_thread (arg=0x7f7151321700) at pthread_create.c:333
6975D:rpzrecurse:#13 0x00007f7160e9d4dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
6976D:rpzrecurse:--------------------------------------------------------------------------------
6977D:rpzrecurse:full backtrace from rpzrecurse/ns3/core.7743 saved in core.7743-backtrace.txt
6978D:rpzrecurse:core dump rpzrecurse/ns3/core.7743 archived as rpzrecurse/ns3/core.7743.gz
6979R:rpzrecurse:FAIL
```December 2020 (9.11.26, 9.11.26-S1, 9.16.10, 9.16.10-S1, 9.17.8)https://gitlab.isc.org/isc-projects/bind9/-/issues/2144double-attach when prefetching2020-09-22T08:53:33ZEvan Huntdouble-attach when prefetching@jtatuya pointed out in a comment on #2122 that there may be an issue with a handle being attached when non-NULL:
> It's just based on code inspection so I'd apologize in advance if I'm mistaken. As a result of this change, `fetchhandl...@jtatuya pointed out in a comment on #2122 that there may be an issue with a handle being attached when non-NULL:
> It's just based on code inspection so I'd apologize in advance if I'm mistaken. As a result of this change, `fetchhandle` structure member was introduced in `ns_client` and it can be passed to `isc_nmhandle_attach` from these three functions:
> - ns_query_recurse
> - query_rpzfetch
> - query_prefetch
>
> Is this safe? If I understand the implementation correctly, `ns_query_recurse` can be called while prefetching is taking place, so I suspect it's possible that `isc_nmhandle_attach` can be called with a non-NULL `ns_client:fetchhandle`. If that can actually happen that would trigger an assertion failure in `isc_nmhandle_attach`.October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.16.8-S1, 9.17.6)Evan HuntEvan Hunthttps://gitlab.isc.org/isc-projects/bind9/-/issues/2143performance loss in 9.16 compared to version 9.112022-08-25T10:25:32ZLaurent Gouhierperformance loss in 9.16 compared to version 9.11# Summary
I was doing some test before upgrading from bind *9.11* to *9.16* and I encounter a performance loss between those versions. I was at *400k query per second in bind 9.11* and *100k query per second in bind 9.16*.
# Hardawre...# Summary
I was doing some test before upgrading from bind *9.11* to *9.16* and I encounter a performance loss between those versions. I was at *400k query per second in bind 9.11* and *100k query per second in bind 9.16*.
# Hardawre Configuration :
- DNS Server ( FreeBSD-11 ):
````
DELL R640
Intel Network Adapter X710
Intel(R) Xeon(R) Gold 5215 CPU @ 2.50GHz 10C/20T
RAM 32G DDR4
````
- Bench ( Debian 10 ) :
````
DELL R340
Intel Network Adapter X710
Intel(R) Xeon(R) E-2136 CPU @ 3.30GHz 6C/12T
RAM 32G DDR4
````
# Network Configuration
````
---------- ------------
| |<-----[ LAN ]-------------------------->| |
| DNS | | BENCH |
| |<-----[ Bench Dedicated Fiber ]-------->| |
--------- 60.0/8 ------------
````
DNS IP for the bench : 60.0.0.1
BENCH IP : 60.0.0.2
# DNS Configuration
named.conf :
````
options {
listen-on-v6 { any; };
version "";
max-cache-size 100M;
max-journal-size 10M;
check-names master ignore;
check-integrity no;
check-sibling no;
dnssec-secure-to-insecure yes;
directory "/etc/namedb";
recursion no;
server-id "bind-auth";
};
controls {
inet 127.0.0.1 port 953 allow {
127.0.0.1/32;
} ;
};
acl "admin" {
"any";
};
zone "incache" {
type master;
file "incache.db";
allow-update { any; };
};
````
incache.db:
````
$ORIGIN @
$TTL 86400 ; 1 day
@ IN SOA bench. root. (
6293 ; serial
21600 ; refresh (6 hours)
3600 ; retry (1 hour)
604800 ; expire (1 week)
600 ; minimum (10 minutes)
)
NS bench
bench A 10.0.22.24
$TTL 86400 ; 1 day
* A 1.1.1.1
* AAAA 8888::1111
````
# BIND version test on FreeBSD-11
BIND 9.16.6 (Stable Release) id:25846cf
````
running on FreeBSD amd64 11.4-STABLE FreeBSD 11.4-STABLE #919 r362186M
built by make with '--enable-threads' '--with-make-clean=no' '--with-gssapi=/usr/local' '--with-geoip' '--prefix=/' '--enable-filter-aaaa' '--enable-fetchlimit'
compiled by CLANG FreeBSD Clang 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2)
compiled with OpenSSL version: OpenSSL 1.1.1g 21 Apr 2020
linked to OpenSSL version: OpenSSL 1.1.1g 21 Apr 2020
compiled with libxml2 version: 2.9.10
linked to libxml2 version: 20910
compiled with libjson-c version: 0.15
linked to libjson-c version: 0.15
compiled with zlib version: 1.2.11
linked to zlib version: 1.2.11
threads support is enabled
````
BIND 9.11.22 (Extended Support Version) id:6a05a96
````
running on FreeBSD amd64 11.4-STABLE FreeBSD 11.4-STABLE #919 r362186M
built by make with '--enable-threads' '--with-make-clean=no' '--with-gssapi=/usr/local' '--with-geoip' '--prefix=/' '--enable-filter-aaaa' '--enable-fetchlimit'
compiled by CLANG FreeBSD Clang 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2)
compiled with OpenSSL version: OpenSSL 1.1.1g 21 Apr 2020
linked to OpenSSL version: OpenSSL 1.1.1g 21 Apr 2020
compiled with libxml2 version: 2.9.10
linked to libxml2 version: 20910
compiled with libjson-c version: 0.15
linked to libjson-c version: 0.15
compiled with zlib version: 1.2.11
linked to zlib version: 1.2.11
threads support is enabled
````
# Software for the Bench
[dnsgen](https://github.com/isc-projects/dnsgen)
# DNSGEN Command Use
dnsgen -i enp1s0f0 -a 60.0.0.2 -s 60.0.0.1 -m f8:f2:f0:f0:f0:f0 -p 53 -d stressfile.dat -r 1000000 -M -R 1 -l 10
[stressfile.dat](/uploads/bfc71a6897fa54beedf5b86aba148513/stressfile.dat)
# Results
Bind 9.16 :
````
DNSGEN :
267251.500128893 1000094 115500 100224 11552
267251.600128893 1000095 115500 99840 11552
267251.700128893 1000096 115520 100224 11584
267251.800128893 1000097 115540 99840 11584
267251.900128893 1000098 115540 100160 11569
267252.000128893 1000099 115520 99904 11525
267252.100128893 1000100 109760 384 44
Peak RX rate = 115540
RCODE 0: 1154189
````
````
TOP :
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
43160 named 63 52 0 254M 128M sigwai 19 0:12 103.44% named{isc-net-0019}
````
Bind 9.11 :
````
DNSGEN :
267411.400128183 1000093 440780 100192 43999
267411.500128183 1000094 441250 99872 44737
267411.600128183 1000095 441430 99840 44303
267411.700128183 1000096 441760 100224 44483
267411.800128183 1000097 442160 99840 44430
267411.900128183 1000098 442380 100160 44404
267412.000128183 1000099 442500 99840 44383
267412.100128183 1000100 420590 384 108
Peak RX rate = 444100
RCODE 0: 4419370
````
````
TOP :
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
44366 named 24 52 0 85584K 36852K sigwai 1 5:00 1659.57% named
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
44366 named 84 0 85584K 36616K CPU10 10 0:05 89.23% named{isc-worker0012}
44366 named 34 0 85584K 36616K CPU19 19 0:01 88.51% named{isc-worker0006}
44366 named 84 0 85584K 36616K CPU12 12 0:05 87.89% named{isc-worker0007}
44366 named 84 0 85584K 36616K CPU8 8 0:05 87.87% named{isc-worker0015}
44366 named 83 0 85584K 36616K RUN 4 0:05 87.76% named{isc-worker0010}
44366 named 83 0 85584K 36616K CPU17 17 0:05 87.35% named{isc-worker0008}
44366 named 84 0 85584K 36616K RUN 1 0:05 87.15% named{isc-worker0001}
44366 named 84 0 85584K 36616K RUN 5 0:05 87.14% named{isc-worker0003}
44366 named 83 0 85584K 36616K CPU9 9 0:05 86.93% named{isc-worker0000}
44366 named 84 0 85584K 36616K CPU14 14 0:05 86.72% named{isc-worker0016}
44366 named 84 0 85584K 36616K CPU7 7 0:05 86.72% named{isc-worker0017}
44366 named 83 0 85584K 36616K CPU11 11 0:05 86.61% named{isc-worker0005}
44366 named 83 0 85584K 36616K CPU16 16 0:05 86.53% named{isc-worker0019}
44366 named 84 0 85584K 36616K CPU6 6 0:05 86.22% named{isc-worker0009}
44366 named 83 0 85584K 36616K CPU15 15 0:05 86.12% named{isc-worker0013}
44366 named 84 0 85584K 36616K RUN 16 0:05 85.84% named{isc-worker0018}
44366 named 83 0 85584K 36616K RUN 2 0:05 85.82% named{isc-worker0004}
44366 named 83 0 85584K 36616K RUN 3 0:05 85.44% named{isc-worker0011}
44366 named 83 0 85584K 36616K CPU13 13 0:05 84.97% named{isc-worker0014}
````
# Conclusion
On the same hardware and on the same OS ( FreeBSD 11 ), I have found a drop of performance between Bind 9.11 and Bind 9.16
On Bind 9.11 we have 400k query per second with ~20 Threads working on the 24 created.
And on Bind 9.16 we have only 100K query per second with 1 Threads working on the 63 created.
I am available to conduct more test if you need to.
Best regards.https://gitlab.isc.org/isc-projects/bind9/-/issues/2142write after free in unit test lib/ns/tests/query_test.c2020-09-24T12:22:17ZMark Andrewswrite after free in unit test lib/ns/tests/query_test.c```
d94dce78c1c4ece876df4f4bd/d476cf377bb844d8f3326c180f2658115041457349f68994da30f5aa0b03b7f3.tsan
WARNING: ThreadSanitizer: heap-use-after-free
Write of size 8 at 0x000000000001 by main thread:
#0 __wrap_isc_nmhandle_detach lib...```
d94dce78c1c4ece876df4f4bd/d476cf377bb844d8f3326c180f2658115041457349f68994da30f5aa0b03b7f3.tsan
WARNING: ThreadSanitizer: heap-use-after-free
Write of size 8 at 0x000000000001 by main thread:
#0 __wrap_isc_nmhandle_detach lib/ns/tests/nstest.c:125:11
#1 ns_test_qctx_destroy lib/ns/tests/nstest.c:874:3
#2 run_sfcache_test lib/ns/tests/query_test.c:163:2
#3 ns__query_sfcache_test lib/ns/tests/query_test.c:245:3
#4 <null> <null>
#5 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16
Previous write of size 8 at 0x000000000001 by main thread (mutexes: write M1):
#0 free <null>
#1 default_memfree lib/isc/mem.c:742:2
#2 mem_put lib/isc/mem.c:654:2
#3 isc___mem_put lib/isc/mem.c:1110:3
#4 isc__mem_put lib/isc/mem.c:2439:2
#5 __wrap_isc_nmhandle_detach lib/ns/tests/nstest.c:122:3
#6 ns_test_qctx_destroy lib/ns/tests/nstest.c:874:3
#7 run_sfcache_test lib/ns/tests/query_test.c:163:2
#8 ns__query_sfcache_test lib/ns/tests/query_test.c:245:3
#9 <null> <null>
#10 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16
Mutex M1 (0x000000000014) created at:
#0 pthread_mutex_init <null>
#1 isc__mutex_init lib/isc/pthreads/mutex.c:288:8
#2 mem_create lib/isc/mem.c:765:2
#3 isc_mem_create lib/isc/mem.c:2425:2
#4 ns_test_begin lib/ns/tests/nstest.c:303:2
#5 _setup lib/ns/tests/query_test.c:43:11
#6 <null> <null>
#7 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16
SUMMARY: ThreadSanitizer: heap-use-after-free lib/ns/tests/nstest.c:125:11 in __wrap_isc_nmhandle_detach
```October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.16.8-S1, 9.17.6)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/2141asynchrony support for BIND 9 query plugins2021-01-15T22:48:55ZJinmei Tatuyaasynchrony support for BIND 9 query plugins### Description
Currently, BIND 9's query plugin framework requires any hook works synchronously. But sometimes a hook action can be time consuming. Consider, for example, a hook that needs to send some kind of query (like a DB lookup...### Description
Currently, BIND 9's query plugin framework requires any hook works synchronously. But sometimes a hook action can be time consuming. Consider, for example, a hook that needs to send some kind of query (like a DB lookup) to an external backend server and waits for the response to complete the hook's action. Right now this has to be synchronous, blocking the BIND 9 worker thread handling the query, which also blocks subsequent DNS queries. It would be much better if it could be asynchronous, similar to the way BIND 9 handles recursive resolution from the query module.
### Request
Based on some experiments it doesn't seem to be difficult to extend it to support asynchronous hook actions. I plan to write a complete patch to implement the idea. This issue is a placeholder for that patch.
### Links / referencesDecember 2020 (9.11.26, 9.11.26-S1, 9.16.10, 9.16.10-S1, 9.17.8)Jinmei TatuyaJinmei Tatuyahttps://gitlab.isc.org/isc-projects/bind9/-/issues/2140convert dig and friends to use the netmgr2022-04-26T13:12:40ZEvan Huntconvert dig and friends to use the netmgrConvert `dig`, `host` and `nslookup` to use the network manager instead of the isc_socket API.Convert `dig`, `host` and `nslookup` to use the network manager instead of the isc_socket API.November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)Evan HuntEvan Hunthttps://gitlab.isc.org/isc-projects/bind9/-/issues/2138big performance loss in 9.16 compared to version 9.112020-09-10T08:48:31ZLaurent Gouhierbig performance loss in 9.16 compared to version 9.11<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please do *NOT* report it here, but send an
email to [...<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please do *NOT* report it here, but send an
email to [security-officer@isc.org](security-officer@isc.org).
-->
### Summary
I was doing some test before upgrading from bind 9.11 to 9.16
and I encounter a big perfomance loss between those verions.
I have a loss of 75% ! I was at 400k query per second in bind 9.11
and in bind 9.16 i was only at 100k query per second
### BIND version used
BIND 9.16.6 (Stable Release) <id:25846cf>
BIND 9.11.22 (Extended Support Version) <id:6a05a96>
### Steps to reproduce
Use the following named.conf file :
````
options {
listen-on-v6 { any; };
version "";
max-cache-size 100M;
max-journal-size 10M;
check-names master ignore;
check-integrity no;
check-sibling no;
dnssec-secure-to-insecure yes;
directory "/etc/namedb";
recursion no;
server-id "bind-auth";
};
controls {
inet 127.0.0.1 port 953 allow {
127.0.0.1/32;
} ;
};
acl "admin" {
"any";
};
zone "incache" {
type master;
file "incache";
allow-update { any; };
};
````
And for the incache zone use the following zone file :
````
$ORIGIN @
$TTL 86400 ; 1 day
@ IN SOA bench. root. (
6293 ; serial
21600 ; refresh (6 hours)
3600 ; retry (1 hour)
604800 ; expire (1 week)
600 ; minimum (10 minutes)
)
NS bench
bench A 10.0.22.24
$TTL 86400 ; 1 day
* A 1.1.1.1
* AAAA 8888::1111
````
Then run a dnsperf (using the file in atachement ):
````
dnsperf -c 2 -l -d file.dnsperf -s $DNS_IP
````
[file.dnsperf](/uploads/aa5b63b63af3f768a42c661976f4b033/file.dnsperf)
### What is the current *bug* behavior?
In the Curent Stable 9.16 on FreeBSD 11-Stable I have 100k query per second
And in the 9.11 on same hardware and same os I have 400k query per second
On a FreeBSD-12 it is better but i still have a performance drop.
On current : 9.16 200k query per second
: 9.11 100k query per second
### What is the expected *correct* behavior?
I was expecting at least to have the same performance on both versions.
### Relevant logs and/or screenshots
DNSPERF on 9.16 : `
````
[Status] Testing complete (time limit)
Statistics:
Queries sent: 966633
Queries completed: 966633 (100.00%)
Queries lost: 0 (0.00%)
Response codes: NOERROR 966633 (100.00%)
Average packet size: request 29, response 45
Run time (s): 10.000892
Queries per second: 96654.678403
Average Latency (s): 0.001024 (min 0.000047, max 0.001866)
Latency StdDev (s): 0.000074
````
DNSPERF on 9.11 :
````
[Status] Testing complete (time limit)
Statistics:
Queries sent: 4144501
Queries completed: 4144501 (100.00%)
Queries lost: 0 (0.00%)
Response codes: NOERROR 4144501 (100.00%)
Average packet size: request 29, response 81
Run time (s): 10.000196
Queries per second: 414441.976937
Average Latency (s): 0.000225 (min 0.000035, max 0.013790)
Latency StdDev (s): 0.000053
````https://gitlab.isc.org/isc-projects/bind9/-/issues/21369.14.10: query.c:8499: INSIST(qctx->is_zone || (((qctx->client)->query.attrib...2020-09-11T12:23:11ZCathy Almond9.14.10: query.c:8499: INSIST(qctx->is_zone || (((qctx->client)->query.attributes & 0x20000) != 0)) failedAll servers that received a zone update removing some NS delegation RRs crashed 5 mins later with :
query.c:8499: INSIST(qctx->is_zone || (((qctx->client)->query.attributes & 0x20000) != 0)) failed
These servers have:
- root hint zone ...All servers that received a zone update removing some NS delegation RRs crashed 5 mins later with :
query.c:8499: INSIST(qctx->is_zone || (((qctx->client)->query.attributes & 0x20000) != 0)) failed
These servers have:
- root hint zone configured (so not a mirrored root zone - although we don't know what's actually in the root hints)
- global forwarding (defaulting to forward first)
- a secondary zone (that received the NS RRs zone update removing some delegation NS RRset)
- (default) 'qname-minimization relaxed;'
Prior to the zone update, the secondary zone contained delegation RRs that would have caused the resolver to follow the delegation path directly (global forwarding overridden in the zone definition).
We do not know if ALL the delegation RRs were removed for each subdomain, or only some of them. We also don't know if there was an exception made to the usual IXFR zone update process this time around.
The zone update should have been received via IXFR and was the removal of 50,000 RRs from a 250,000 RR zone.
Post zone-update, all the servers that crashed received a client query that should have either been answered from, or followed delegation from, the zone that had just been updated.
This crash location looks very much like #1862, but the scenario is a little different, due to the lack of mirrored root.
We have core dumps, binaries and libs available - please see [Support Ticket #17077](https://support.isc.org/Ticket/Display.html?id=17077) for further customer-confidential details.
**NOTE: This is an important issue for a high-status ISC customer. Although this was encountered on 9.14.10, assurance is needed that the underlying cause is identified and has (or will be soon) fixed.**Witold KrecickiWitold Krecickihttps://gitlab.isc.org/isc-projects/bind9/-/issues/2135Documentation suggestion for pkcs11-keygen, pkcs11-list ARM documentation2021-10-05T15:33:49ZMichael McNallyDocumentation suggestion for pkcs11-keygen, pkcs11-list ARM documentationA customer who struggled with this a bit while trying to get our native pkcs11 implementation and its tools working, suggests (via their [Support ticket](https://support.isc.org/Ticket/Display.html?id=15826)) that it might help users of ...A customer who struggled with this a bit while trying to get our native pkcs11 implementation and its tools working, suggests (via their [Support ticket](https://support.isc.org/Ticket/Display.html?id=15826)) that it might help users of our provided tools if we were more explicit / clearer about when the slot must be specified with a -s command-line argument:
> With your permission, I would like to remark:
> In the [Bind ARM](https://ftp.isc.org/isc/bind9/cur/9.11/doc/arm/Bv9ARM.ch04.html#pkcs11) it would be better to note that when using pksc11-list and pkcs11-keygen commands it may be needed to define the slot.
>
> otherwise the error..
> ```
> pkcs11-keygen -b 2048 -l sample-ksk
> Enter Pin:
> Unrecoverable error initializing PKCS#11: not found
>
> pkcs11-list -s 12129377564
> slot 12129377564
> Enter Pin:
> object[0]: handle 2 class 2 label[10] 'sample-ksk' id[0]
> object[1]: handle 3 class 3 label[10] 'sample-ksk' id[0] E:never
> ```
> same with pkcs11-list commandhttps://gitlab.isc.org/isc-projects/bind9/-/issues/2134isc_nm_closedown seems to not close all connections2020-12-03T09:54:54ZOndřej Surýisc_nm_closedown seems to not close all connectionsWhen the `isccc_ccmsg_init()` is changed to attach to `isc_nmhandle_t` and `isccc_ccmsg_invalidate()` is changed to detach from `isc_nmhandle_t`, the code either:
* hangs
* produces following backtrace on macOS:
```
ccmsg.c:48: INSIST((...When the `isccc_ccmsg_init()` is changed to attach to `isc_nmhandle_t` and `isccc_ccmsg_invalidate()` is changed to detach from `isc_nmhandle_t`, the code either:
* hangs
* produces following backtrace on macOS:
```
ccmsg.c:48: INSIST((__builtin_expect(!!((ccmsg) != ((void*)0)), 1) && __builtin_expect(!!(((const isc__magic_t *)(ccmsg))->magic == ((('C') << 24 | ('C') << 16 | ('m') << 8 | ('s')))), 1))) failed, back trace
0 libisc.1704.dylib 0x0000000109c960fa default_callback + 74
1 libisc.1704.dylib 0x0000000109c9608a isc_assertion_failed + 10
2 libisccc.1702.dylib 0x0000000109f21e4e recv_data + 430
3 libisc.1704.dylib 0x0000000109c81a04 isc__nm_tcp_shutdown + 68
4 libuv.1.dylib 0x0000000109fe9f9a uv_walk + 151
5 libisc.1704.dylib 0x0000000109c7f13d process_queue + 413
6 libuv.1.dylib 0x0000000109feab68 uv__async_io + 279
7 libuv.1.dylib 0x0000000109ffa7f2 uv__io_poll + 1882
8 libuv.1.dylib 0x0000000109feaf9e uv_run + 359
9 libisc.1704.dylib 0x0000000109c7c1c5 nm_thread + 53
10 libsystem_pthread.dylib 0x00007fff6aa01109 _pthread_start + 148
11 libsystem_pthread.dylib 0x00007fff6a9fcb8b thread_start + 15
```
which strongly suggests that `isc_nm_closedown()` actually doesn't close all the active connections, and there's interaction between the last call to `isc_nmhandle_detach()` in `isccc_ccmsg_invalidate()` and the callback. All the callbacks must be called before `isc_nm_closedown()` returns and it doesn't seem to be the case here.December 2020 (9.11.26, 9.11.26-S1, 9.16.10, 9.16.10-S1, 9.17.8)https://gitlab.isc.org/isc-projects/bind9/-/issues/2132BIND 9.11 fails to build on FreeBSD with devel/jsoncpp installed2020-09-08T08:56:29ZMichal NowakBIND 9.11 fails to build on FreeBSD with devel/jsoncpp installed`9.11` (but not `9_16` and `main`) fails to build on FreeBSD when `devel/jsoncpp` is installed and JSON is enabled in BIND:
```
--- app.o ---
In file included from app.c:33:
In file included from ../include/isc/mem.h:20:
In file included...`9.11` (but not `9_16` and `main`) fails to build on FreeBSD when `devel/jsoncpp` is installed and JSON is enabled in BIND:
```
--- app.o ---
In file included from app.c:33:
In file included from ../include/isc/mem.h:20:
In file included from ../include/isc/json.h:36:
In file included from /usr/local/include/json/json.h:9:
/usr/local/include/json/config.h:8:10: fatal error: 'cstddef' file not found
#include <cstddef>
^~~~~~~~~
```
For the time being I resolved this locally by uninstalling `devel/jsoncpp` port.
Here's downstream bug report (for 9.14) https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=243387 with a link to their solution.October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.16.8-S1, 9.17.6)https://gitlab.isc.org/isc-projects/bind9/-/issues/2130ThreadSanitizer: heap-use-after-free lib/ns/tests/nstest.c:125:11 in __wrap_i...2020-09-17T08:12:49ZOndřej SurýThreadSanitizer: heap-use-after-free lib/ns/tests/nstest.c:125:11 in __wrap_isc_nmhandle_detachHappens in !4089 MR:
```
WARNING: ThreadSanitizer: heap-use-after-free
Write of size 8 at 0x000000000001 by main thread:
#0 __wrap_isc_nmhandle_detach lib/ns/tests/nstest.c:125:11
#1 ns_test_qctx_destroy lib/ns/tests/nstest.c:8...Happens in !4089 MR:
```
WARNING: ThreadSanitizer: heap-use-after-free
Write of size 8 at 0x000000000001 by main thread:
#0 __wrap_isc_nmhandle_detach lib/ns/tests/nstest.c:125:11
#1 ns_test_qctx_destroy lib/ns/tests/nstest.c:872:3
#2 run_sfcache_test lib/ns/tests/query_test.c:163:2
#3 ns__query_sfcache_test lib/ns/tests/query_test.c:245:3
#4 cmocka_run_one_test_or_fixture <null>
#5 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16
Previous write of size 8 at 0x000000000001 by main thread (mutexes: write M1):
#0 free <null>
#1 default_memfree lib/isc/mem.c:742:2
#2 mem_put lib/isc/mem.c:654:2
#3 isc___mem_put lib/isc/mem.c:1110:3
#4 isc__mem_put lib/isc/mem.c:2439:2
#5 __wrap_isc_nmhandle_detach lib/ns/tests/nstest.c:122:3
#6 ns_test_qctx_destroy lib/ns/tests/nstest.c:872:3
#7 run_sfcache_test lib/ns/tests/query_test.c:163:2
#8 ns__query_sfcache_test lib/ns/tests/query_test.c:245:3
#9 cmocka_run_one_test_or_fixture <null>
#10 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16
Mutex M1 (0x000000000014) created at:
#0 pthread_mutex_init <null>
#1 isc__mutex_init lib/isc/pthreads/mutex.c:288:8
#2 mem_create lib/isc/mem.c:765:2
#3 isc_mem_create lib/isc/mem.c:2425:2
#4 ns_test_begin lib/ns/tests/nstest.c:303:2
#5 _setup lib/ns/tests/query_test.c:43:11
#6 cmocka_run_one_test_or_fixture <null>
#7 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16
SUMMARY: ThreadSanitizer: heap-use-after-free lib/ns/tests/nstest.c:125:11 in __wrap_isc_nmhandle_detach
```October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.16.8-S1, 9.17.6)Evan HuntEvan Hunt