BIND issueshttps://gitlab.isc.org/isc-projects/bind9/-/issues2024-02-24T08:19:32Zhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4427Various improvements to hashing and hash table management2024-02-24T08:19:32ZMichał KępieńVarious improvements to hashing and hash table managementThis is a meta issue to keep track of various improvements to hashing
and hash table management that were implemented since ~"v9.18".
Sparked by a [Mattermost discussion][1].
---
- [x] #4306/!8288 Implement incremental hashing
---
...This is a meta issue to keep track of various improvements to hashing
and hash table management that were implemented since ~"v9.18".
Sparked by a [Mattermost discussion][1].
---
- [x] #4306/!8288 Implement incremental hashing
---
[1]: https://mattermost.isc.org/isc/pl/rsyemrkwhtfcbxhtyxddhkn58yMay 2024 (9.18.27, 9.18.27-S1, 9.19.24)https://gitlab.isc.org/isc-projects/bind9/-/issues/4586Don't count expired / future RRSIGs in verification failure quota2024-02-24T08:17:15ZMark AndrewsDon't count expired / future RRSIGs in verification failure quotaExpired / future RRSIGs don't trigger a public key verification.Expired / future RRSIGs don't trigger a public key verification.May 2024 (9.18.27, 9.18.27-S1, 9.19.24)https://gitlab.isc.org/isc-projects/bind9/-/issues/4352inline-signing breaks nsdiff2024-02-24T07:55:39ZBjörn Perssoninline-signing breaks nsdiff### Summary
When both inline-signing and update-policy are in use, I can't detect race conditions with the method described in RFC 2136 section 5.7, which nsdiff uses.
In a zone that uses dnssec-policy and relies on the default value o...### Summary
When both inline-signing and update-policy are in use, I can't detect race conditions with the method described in RFC 2136 section 5.7, which nsdiff uses.
In a zone that uses dnssec-policy and relies on the default value of inline-signing, the method in RFC 2136 section 5.7 will stop working on upgrade to BIND 9.20, as inline-signing will then be switched on by default, if I understand correctly.
### BIND version used
```
$ named -V
BIND 9.18.19-1~deb12u1-Debian (Extended Support Version) <id:>
running on Linux x86_64 5.10.0-25-amd64 #1 SMP Debian 5.10.191-1 (2023-08-16)
built by make with '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/x86_64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--disable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=yes' '--with-libidn2' '--with-json-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' '--enable-dnstap' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -ffile-prefix-map=/build/reproducible-path/bind9-9.18.19=. -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
compiled by GCC 12.2.0
compiled with OpenSSL version: OpenSSL 3.0.10 1 Aug 2023
linked to OpenSSL version: OpenSSL 3.0.9 30 May 2023
compiled with libuv version: 1.44.2
linked to libuv version: 1.44.2
compiled with libnghttp2 version: 1.52.0
linked to libnghttp2 version: 1.52.0
compiled with libxml2 version: 2.9.14
linked to libxml2 version: 20914
compiled with json-c version: 0.16
linked to json-c version: 0.16
compiled with zlib version: 1.2.13
linked to zlib version: 1.2.13
linked to maxminddb version: 1.7.1
compiled with protobuf-c version: 1.4.1
linked to protobuf-c version: 1.4.1
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): yes
default paths:
named configuration: /etc/bind/named.conf
rndc configuration: /etc/bind/rndc.conf
DNSSEC root key: /etc/bind/bind.keys
nsupdate session key: //run/named/session.key
named PID file: //run/named/named.pid
named lock file: //run/named/named.lock
geoip-directory: /usr/share/GeoIP
```
### Steps to reproduce
Here I start from a working state with serial number 2023091800. The prerequisite matches the reply to the SOA query, and the update is answered with NOERROR. This is correct as far as I understand:
```
$ (echo 'prereq yxrrset xn--rombobjrn-67a.se. IN SOA ns1.xn--rombobjrn-67a.se. hostmaster.xn--rombobjrn-67a.se. 2023091800 14400 3600 3024000 86400' ; echo send) | nsupdate -k internal -d
Creating key...
Creating key...
namefromtext
keycreate
Reply from SOA query:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23595
;; flags: qr aa rd ra; QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;xn--rombobjrn-67a.se. IN SOA
;; ANSWER SECTION:
xn--rombobjrn-67a.se. 86400 IN SOA ns1.xn--rombobjrn-67a.se. hostmaster.xn--rombobjrn-67a.se. 2023091800 14400 3600 3024000 86400
Found zone name: xn--rombobjrn-67a.se
The primary is: ns1.xn--rombobjrn-67a.se
Sending update to 192.168.72.1#53
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 44980
;; flags:; ZONE: 1, PREREQ: 1, UPDATE: 0, ADDITIONAL: 1
;; PREREQUISITE SECTION:
xn--rombobjrn-67a.se. 0 IN SOA ns1.xn--rombobjrn-67a.se. hostmaster.xn--rombobjrn-67a.se. 2023091800 14400 3600 3024000 86400
;; TSIG PSEUDOSECTION:
internal.beorn.tag.xn--rombobjrn-67a.se. 0 ANY TSIG hmac-sha512. 1695171088 300 64 [...] 44980 NOERROR 0
Reply from update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 44980
;; flags: qr; ZONE: 1, PREREQ: 0, UPDATE: 0, ADDITIONAL: 1
;; ZONE SECTION:
;xn--rombobjrn-67a.se. IN SOA
;; TSIG PSEUDOSECTION:
internal.beorn.tag.xn--rombobjrn-67a.se. 0 ANY TSIG hmac-sha512. 1695171088 300 64 [...] 44980 NOERROR 0
```
Later, the server has automatically increased the serial number to 2023091802. I use the new serial number in the prerequisite, so it looks identical to the new SOA value, but this time the update is rejected with NXRRSET:
```
$ (echo 'prereq yxrrset xn--rombobjrn-67a.se. IN SOA ns1.xn--rombobjrn-67a.se. hostmaster.xn--rombobjrn-67a.se. 2023091802 14400 3600 3024000 86400' ; echo send) | nsupdate -k internal -d
Creating key...
Creating key...
namefromtext
keycreate
Reply from SOA query:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65052
;; flags: qr aa rd ra; QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;xn--rombobjrn-67a.se. IN SOA
;; ANSWER SECTION:
xn--rombobjrn-67a.se. 86400 IN SOA ns1.xn--rombobjrn-67a.se. hostmaster.xn--rombobjrn-67a.se. 2023091802 14400 3600 3024000 86400
Found zone name: xn--rombobjrn-67a.se
The primary is: ns1.xn--rombobjrn-67a.se
Sending update to 192.168.72.1#53
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 52912
;; flags:; ZONE: 1, PREREQ: 1, UPDATE: 0, ADDITIONAL: 1
;; PREREQUISITE SECTION:
xn--rombobjrn-67a.se. 0 IN SOA ns1.xn--rombobjrn-67a.se. hostmaster.xn--rombobjrn-67a.se. 2023091802 14400 3600 3024000 86400
;; TSIG PSEUDOSECTION:
internal.beorn.tag.xn--rombobjrn-67a.se. 0 ANY TSIG hmac-sha512. 1695259647 300 64 [...] 52912 NOERROR 0
Reply from update query:
;; ->>HEADER<<- opcode: UPDATE, status: NXRRSET, id: 52912
;; flags: qr; ZONE: 1, PREREQ: 0, UPDATE: 0, ADDITIONAL: 1
;; ZONE SECTION:
;xn--rombobjrn-67a.se. IN SOA
;; TSIG PSEUDOSECTION:
internal.beorn.tag.xn--rombobjrn-67a.se. 0 ANY TSIG hmac-sha512. 1695259647 300 64 [...] 52912 NOERROR 0
```
Now I change the prerequisite back to 2023091800. The SOA hasn't changed again. The serial number is still 2023091802. This update should be rejected as the prerequisite contains an outdated serial number, but is in fact answered with NOERROR:
```
$ (echo 'prereq yxrrset xn--rombobjrn-67a.se. IN SOA ns1.xn--rombobjrn-67a.se. hostmaster.xn--rombobjrn-67a.se. 2023091800 14400 3600 3024000 86400' ; echo send) | nsupdate -k internal -d
Creating key...
Creating key...
namefromtext
keycreate
Reply from SOA query:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6226
;; flags: qr aa rd ra; QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;xn--rombobjrn-67a.se. IN SOA
;; ANSWER SECTION:
xn--rombobjrn-67a.se. 86400 IN SOA ns1.xn--rombobjrn-67a.se. hostmaster.xn--rombobjrn-67a.se. 2023091802 14400 3600 3024000 86400
Found zone name: xn--rombobjrn-67a.se
The primary is: ns1.xn--rombobjrn-67a.se
Sending update to 192.168.72.1#53
Outgoing update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 49961
;; flags:; ZONE: 1, PREREQ: 1, UPDATE: 0, ADDITIONAL: 1
;; PREREQUISITE SECTION:
xn--rombobjrn-67a.se. 0 IN SOA ns1.xn--rombobjrn-67a.se. hostmaster.xn--rombobjrn-67a.se. 2023091800 14400 3600 3024000 86400
;; TSIG PSEUDOSECTION:
internal.beorn.tag.xn--rombobjrn-67a.se. 0 ANY TSIG hmac-sha512. 1695263626 300 64 [...] 49961 NOERROR 0
Reply from update query:
;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 49961
;; flags: qr; ZONE: 1, PREREQ: 0, UPDATE: 0, ADDITIONAL: 1
;; ZONE SECTION:
;xn--rombobjrn-67a.se. IN SOA
;; TSIG PSEUDOSECTION:
internal.beorn.tag.xn--rombobjrn-67a.se. 0 ANY TSIG hmac-sha512. 1695263626 300 64 [...] 49961 NOERROR 0
```
### What is the current *bug* behavior?
It seems that a serial number specified in a prerequisite of an update is compared to the unsigned version of the zone, but the serial number retrieved with a SOA or AXFR query is from the signed version. As far as I know a client can't look up the unsigned serial number, and thus can't specify it in the prerequisite. Thus the update fails when the two serial numbers differ.
### What is the expected *correct* behavior?
It seems to me that a prerequisite that specifies a SOA record should be checked against the same record that a client gets in response to a SOA or AXFR query. I don't know what other usecases that might break though.
### Relevant excerpts from the configuration file
```
dnssec-policy "some_name" {
keys {
ksk lifetime unlimited algorithm rsasha256 2048;
zsk lifetime unlimited algorithm rsasha256 2048;
};
dnskey-ttl P1D;
purge-keys 0;
};
view "internal" {
allow-transfer { key internal.beorn.tag.xn--rombobjrn-67a.se.; };
zone "xn--rombobjrn-67a.se" {
type master;
file "/var/lib/bind/db.xn--rombobjrn-67a.se.internal";
dnssec-policy some_name;
parental-agents { ::1; };
inline-signing yes;
update-policy {
grant internal.beorn.tag.xn--rombobjrn-67a.se. zonesub ANY;
};
};
};May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/4333catz: assertion failure in dns_view_attach() during shutdown2024-02-24T07:55:32ZArаm Sаrgsyаncatz: assertion failure in dns_view_attach() during shutdownSee https://gitlab.isc.org/isc-projects/bind9/-/jobs/3673976
This has happened on a branch based on `main`, so for now only ~"Affects v9.19" is set, but the other branches are probably affected too. The labels will updated when there is...See https://gitlab.isc.org/isc-projects/bind9/-/jobs/3673976
This has happened on a branch based on `main`, so for now only ~"Affects v9.19" is set, but the other branches are probably affected too. The labels will updated when there is more information.
```
[New LWP 55771]
[New LWP 55766]
[New LWP 55752]
[New LWP 55767]
[New LWP 55772]
[New LWP 55770]
[New LWP 55773]
[New LWP 55769]
[New LWP 55768]
[New LWP 55774]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/builds/isc-projects/bind9/bin/named/.libs/lt-named -D catz_tmp_b13wq61e-ns4 -X'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f0c573e0b8f in raise () from /lib64/libc.so.6
[Current thread is 1 (Thread 0x7f0c3c87e700 (LWP 55771))]
Thread 10 (Thread 0x7f0c3b07b700 (LWP 55774)):
#0 0x00007f0c573cb9bd in syscall () from /lib64/libc.so.6
No symbol table info available.
#1 0x00007f0c57b68c76 in synchronize_rcu_memb () from /lib64/liburcu.so.6
No symbol table info available.
#2 0x00007f0c57b6975d in call_rcu_thread () from /lib64/liburcu.so.6
No symbol table info available.
#3 0x00007f0c580f81da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#4 0x00007f0c573cbe73 in clone () from /lib64/libc.so.6
No symbol table info available.
Thread 9 (Thread 0x7f0c5247d700 (LWP 55768)):
#0 0x00007f0c580ff60e in pthread_barrier_wait () from /lib64/libpthread.so.0
No symbol table info available.
#1 0x00007f0c5b8416f9 in resume_loop (loop=0x7f0c53e93180) at loop.c:114
loopmgr = <optimized out>
loopmgr = <optimized out>
#2 pauseresume_cb (handle=<optimized out>) at loop.c:114
loop = 0x7f0c53e93180
#3 0x00007f0c5922a2f1 in uv.async_io.part () from /lib64/libuv.so.1
No symbol table info available.
#4 0x00007f0c5923bd15 in uv.io_poll () from /lib64/libuv.so.1
No symbol table info available.
#5 0x00007f0c5922aa74 in uv_run () from /lib64/libuv.so.1
No symbol table info available.
#6 0x00007f0c5b841792 in loop_thread (arg=arg@entry=0x7f0c53e93180) at loop.c:282
loop = 0x7f0c53e93180
r = <optimized out>
__func__ = "loop_thread"
ret = <optimized out>
#7 0x00007f0c5b850469 in thread_body (wrap=wrap@entry=0xf19630) at thread.c:85
func = 0x7f0c5b841707 <loop_thread>
arg = 0x7f0c53e93180
ret = 0x0
jemalloc_enforce_init = 0x7f0c48000b60
#8 0x00007f0c5b850492 in thread_run (wrap=0xf19630) at thread.c:100
ret = <optimized out>
#9 0x00007f0c580f81da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#10 0x00007f0c573cbe73 in clone () from /lib64/libc.so.6
No symbol table info available.
Thread 8 (Thread 0x7f0c51c7c700 (LWP 55769)):
#0 0x00007f0c573cb9bd in syscall () from /lib64/libc.so.6
No symbol table info available.
#1 0x00007f0c5795e96d in futex_wait.part () from /lib64/liburcu-cds.so.6
No symbol table info available.
#2 0x00007f0c5795ee10 in workqueue_thread () from /lib64/liburcu-cds.so.6
No symbol table info available.
#3 0x00007f0c580f81da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#4 0x00007f0c573cbe73 in clone () from /lib64/libc.so.6
No symbol table info available.
Thread 7 (Thread 0x7f0c3b87c700 (LWP 55773)):
#0 0x00007f0c5810187d in __lll_lock_wait () from /lib64/libpthread.so.0
No symbol table info available.
#1 0x00007f0c580fab29 in pthread_mutex_lock () from /lib64/libpthread.so.0
No symbol table info available.
#2 0x00007f0c57b681c9 in mutex_lock () from /lib64/liburcu.so.6
No symbol table info available.
#3 0x00007f0c57b69498 in rcu_register_thread_memb () from /lib64/liburcu.so.6
No symbol table info available.
#4 0x00007f0c5b856969 in isc__work_cb (req=<optimized out>) at work.c:28
work = 0x7f0c406203c0
#5 0x00007f0c592254ee in worker () from /lib64/libuv.so.1
No symbol table info available.
#6 0x00007f0c580f81da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#7 0x00007f0c573cbe73 in clone () from /lib64/libc.so.6
No symbol table info available.
Thread 6 (Thread 0x7f0c3d07f700 (LWP 55770)):
#0 0x00007f0c580fe4ac in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
No symbol table info available.
#1 0x00007f0c5923821d in uv_cond_wait () from /lib64/libuv.so.1
No symbol table info available.
#2 0x00007f0c5922558d in worker () from /lib64/libuv.so.1
No symbol table info available.
#3 0x00007f0c580f81da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#4 0x00007f0c573cbe73 in clone () from /lib64/libc.so.6
No symbol table info available.
Thread 5 (Thread 0x7f0c3c07d700 (LWP 55772)):
#0 0x00007f0c580fe4ac in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
No symbol table info available.
#1 0x00007f0c5923821d in uv_cond_wait () from /lib64/libuv.so.1
No symbol table info available.
#2 0x00007f0c5922558d in worker () from /lib64/libuv.so.1
No symbol table info available.
#3 0x00007f0c580f81da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#4 0x00007f0c573cbe73 in clone () from /lib64/libc.so.6
No symbol table info available.
Thread 4 (Thread 0x7f0c52c7e700 (LWP 55767)):
#0 0x00007f0c580ff60e in pthread_barrier_wait () from /lib64/libpthread.so.0
No symbol table info available.
#1 0x00007f0c5b8416f9 in resume_loop (loop=0x7f0c53e92900) at loop.c:114
loopmgr = <optimized out>
loopmgr = <optimized out>
#2 pauseresume_cb (handle=<optimized out>) at loop.c:114
loop = 0x7f0c53e92900
#3 0x00007f0c5922a2f1 in uv.async_io.part () from /lib64/libuv.so.1
No symbol table info available.
#4 0x00007f0c5923bd15 in uv.io_poll () from /lib64/libuv.so.1
No symbol table info available.
#5 0x00007f0c5922aa74 in uv_run () from /lib64/libuv.so.1
No symbol table info available.
#6 0x00007f0c5b841792 in loop_thread (arg=arg@entry=0x7f0c53e92900) at loop.c:282
loop = 0x7f0c53e92900
r = <optimized out>
__func__ = "loop_thread"
ret = <optimized out>
#7 0x00007f0c5b850469 in thread_body (wrap=wrap@entry=0xf19660) at thread.c:85
func = 0x7f0c5b841707 <loop_thread>
arg = 0x7f0c53e92900
ret = 0x0
jemalloc_enforce_init = 0x7f0c44000b60
#8 0x00007f0c5b850492 in thread_run (wrap=0xf19660) at thread.c:100
ret = <optimized out>
#9 0x00007f0c580f81da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#10 0x00007f0c573cbe73 in clone () from /lib64/libc.so.6
No symbol table info available.
Thread 3 (Thread 0x7f0c5bd5e480 (LWP 55752)):
#0 0x00007f0c5810187d in __lll_lock_wait () from /lib64/libpthread.so.0
No symbol table info available.
#1 0x00007f0c580fab29 in pthread_mutex_lock () from /lib64/libpthread.so.0
No symbol table info available.
#2 0x00007f0c57b681c9 in mutex_lock () from /lib64/liburcu.so.6
No symbol table info available.
#3 0x00007f0c57b68c44 in synchronize_rcu_memb () from /lib64/liburcu.so.6
No symbol table info available.
#4 0x00007f0c5b36f3ca in dns_view_detach (viewp=viewp@entry=0x7ffddd6d9348) at view.c:519
mkzone = 0x0
rdzone = 0x0
resolver = 0x0
adb = 0x7f0c40625600
zonetable = 0x7f0c5395d0a0
requestmgr = 0x7f0c4061b1e0
dispatchmgr = 0x7f0c53e7a9a0
view = 0x7f0c538cfc00
__func__ = "dns_view_detach"
#5 0x00000000004457e1 in shutdown_server (arg=0x7f0c53e881c0) at server.c:10025
server = 0x7f0c53e881c0
view = 0x0
view_next = 0x7f0c538cf600
kasp = 0x0
kasp_next = <optimized out>
flush = false
nsc = 0x0
#6 0x00007f0c5b82e23a in isc__async_cb (handle=<optimized out>) at async.c:111
job = 0x7f0c53e673c0
loop = 0x7f0c53e91800
jobs = {head = {node = {next = 0x7f0c53e63cd0}}, tail = {p = 0x7f0c53e673d0}}
ret = <optimized out>
node = 0x7f0c53e673d0
next = 0x0
#7 0x00007f0c5922a2f1 in uv.async_io.part () from /lib64/libuv.so.1
No symbol table info available.
#8 0x00007f0c5923bd15 in uv.io_poll () from /lib64/libuv.so.1
No symbol table info available.
#9 0x00007f0c5922aa74 in uv_run () from /lib64/libuv.so.1
No symbol table info available.
#10 0x00007f0c5b841792 in loop_thread (arg=arg@entry=0x7f0c53e91800) at loop.c:282
loop = 0x7f0c53e91800
r = <optimized out>
__func__ = "loop_thread"
ret = <optimized out>
#11 0x00007f0c5b850469 in thread_body (wrap=0xf1a3e0) at thread.c:85
func = 0x7f0c5b841707 <loop_thread>
arg = 0x7f0c53e91800
ret = 0x0
jemalloc_enforce_init = 0xf1a410
#12 0x00007f0c5b8504e3 in isc_thread_main (func=func@entry=0x7f0c5b841707 <loop_thread>, arg=0x7f0c53e91800) at thread.c:116
No locals.
#13 0x00007f0c5b84271a in isc_loopmgr_run (loopmgr=0x7f0c53e11540) at loop.c:454
__func__ = "isc_loopmgr_run"
#14 0x0000000000425995 in main (argc=<optimized out>, argv=<optimized out>) at main.c:1580
result = <optimized out>
Thread 2 (Thread 0x7f0c5347f700 (LWP 55766)):
#0 0x00007f0c580ff60e in pthread_barrier_wait () from /lib64/libpthread.so.0
No symbol table info available.
#1 0x00007f0c5b8416f9 in resume_loop (loop=0x7f0c53e92080) at loop.c:114
loopmgr = <optimized out>
loopmgr = <optimized out>
#2 pauseresume_cb (handle=<optimized out>) at loop.c:114
loop = 0x7f0c53e92080
#3 0x00007f0c5922a2f1 in uv.async_io.part () from /lib64/libuv.so.1
No symbol table info available.
#4 0x00007f0c5923bd15 in uv.io_poll () from /lib64/libuv.so.1
No symbol table info available.
#5 0x00007f0c5922aa74 in uv_run () from /lib64/libuv.so.1
No symbol table info available.
#6 0x00007f0c5b841792 in loop_thread (arg=arg@entry=0x7f0c53e92080) at loop.c:282
loop = 0x7f0c53e92080
r = <optimized out>
__func__ = "loop_thread"
ret = <optimized out>
#7 0x00007f0c5b850469 in thread_body (wrap=wrap@entry=0xf19710) at thread.c:85
func = 0x7f0c5b841707 <loop_thread>
arg = 0x7f0c53e92080
ret = 0x0
jemalloc_enforce_init = 0x7f0c4c000b60
#8 0x00007f0c5b850492 in thread_run (wrap=0xf19710) at thread.c:100
ret = <optimized out>
#9 0x00007f0c580f81da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#10 0x00007f0c573cbe73 in clone () from /lib64/libc.so.6
No symbol table info available.
Thread 1 (Thread 0x7f0c3c87e700 (LWP 55771)):
#0 0x00007f0c573e0b8f in raise () from /lib64/libc.so.6
No symbol table info available.
#1 0x00007f0c573b3ea5 in abort () from /lib64/libc.so.6
No symbol table info available.
#2 0x0000000000422efd in assertion_failed (file=0x7f0c5b3eb932 "view.c", line=429, type=isc_assertiontype_insist, cond=0x7f0c5b3c1f70 "__v > 0 && __v < (4294967295U)") at main.c:234
No locals.
#3 0x00007f0c5b82df3b in isc_assertion_failed (file=file@entry=0x7f0c5b3eb932 "view.c", line=line@entry=429, type=type@entry=isc_assertiontype_insist, cond=cond@entry=0x7f0c5b3c1f70 "__v > 0 && __v < (4294967295U)") at assertions.c:48
No locals.
#4 0x00007f0c5b36c3f1 in dns_view_attach (source=source@entry=0x7f0c538cfc00, targetp=targetp@entry=0x7f0c3a20e4c8) at view.c:431
__v = <optimized out>
#5 0x000000000042ba15 in catz_run (entry=0x7f0c39e25000, origin=origin@entry=0x7f0c53e88540, view=0x7f0c538cfc00, udata=0x680670 <ns_catz_cbdata>, type=type@entry=CATZ_DELZONE) at server.c:2956
cz = 0x7f0c3a20e4b0
action = 0x42baf4 <catz_delzone_cb>
#6 0x000000000042ba6e in catz_delzone (entry=<optimized out>, origin=origin@entry=0x7f0c53e88540, view=<optimized out>, udata=<optimized out>) at server.c:2972
No locals.
#7 0x00007f0c5b244c2b in dns__catz_zones_merge (catz=0x7f0c53e88540, newcatz=0x7f0c3a211000) at catz.c:696
entry = 0x7f0c39e25000
result = ISC_R_SUCCESS
iter1 = 0x0
iter2 = 0x7f0c3a21e060
iteradd = 0x7f0c3a21e040
itermod = 0x7f0c3a21e020
toadd = 0x7f0c53e6aa30
tomod = 0x7f0c53e6a9e0
delcur = <optimized out>
czname = "catalog-tls.example\000\f\177\000\000\240\340tW\f\177\000\000\000\000\000\000\000\000\000\000\212\067HW\f\177\000\000@\000\000\000\000\000\000\000\000\006\"s\223>Z\177 \260\207<\f\177\000\000\034\024\205[\f\177\000\000\067\000\000\000\037\000\000\000\016\000\000\000\026\000\000\000\b\000\000\000{\000\000\000\001\000\000\000\000\000\000\000P\262\207<\f\177\000\000\365\016\204[\f\177\000\000{e\206[\f\177\000\000\000&uW\f\177\000\000\240\343tW\f\177\000\000Pl\205[\f\177\000\000{e\206[\f\177\000\000\000\000\000\000\000\000\000\000P\262\207<\f\177\000\000\300\b\204[\f\177\000\000H\361\350S\f\177\000\000"...
zname = "tls1.example\000\177\000\000\377", '\000' <repeats 23 times>, '\377' <repeats 16 times>, '\000' <repeats 24 times>, "`\330tW\f\177\000\000\000\006\"s\223>Z\177\000&uW\f\177\000\000\207(\255\373\000\000\000\000\360\200\346S\f\177\000\000\060\264\207<\f\177\000\000!fuB\000\000\000\000\240\253\207<\f\177\000\000\377", '\000' <repeats 23 times>, '\377' <repeats 16 times>, "\000\006\"s\223>Z\177\340\253\207<\f\177\000\000\000"...
addzone = 0x42ba81 <catz_addzone>
modzone = 0x42ba70 <catz_modzone>
delzone = 0x42ba5f <catz_delzone>
__func__ = "dns__catz_zones_merge"
#8 0x00007f0c5b248106 in dns__catz_update_cb (data=<optimized out>) at catz.c:2467
catz = <optimized out>
updb = <optimized out>
catzs = <optimized out>
oldcatz = 0x7f0c53e88540
newcatz = 0x7f0c3a211000
result = ISC_R_SHUTTINGDOWN
r = <optimized out>
node = 0x0
vers_node = 0x0
updbit = 0x0
fixname = {name = {magic = 1145983854, ndata = 0x0, length = 0, labels = 0, attributes = {absolute = false, readonly = false, dynamic = false, dynoffsets = false, nocompress = false, cache = false, answer = false, ncache = false, chaining = false, chase = false, wildcard = false, prerequisite = false, update = false, hasupdaterec = false}, offsets = 0x7f0c3c87bf40 "", buffer = 0x7f0c3c87bfc0, link = {prev = 0xffffffffffffffff, next = 0xffffffffffffffff}, list = {head = 0x0, tail = 0x0}}, offsets = "\000\b\024\034", '\000' <repeats 123 times>, buffer = {magic = 1114990113, base = 0x7f0c3c87c000, length = 255, used = 0, current = 0, active = 0, extra = 0, dynamic = false, link = {prev = 0xffffffffffffffff, next = 0xffffffffffffffff}, mctx = 0x0}, data = "\aversion\vcatalog-tls\aexample", '\000' <repeats 36 times>, "!fuB\000\000\000\000@y\":\f\177\000\000\000\000\002\000\016", '\000' <repeats 19 times>, '\377' <repeats 16 times>, "\000\000\000\000\000\000\000\000!fuB", '\000' <repeats 20 times>, ")\253\017X\f\177\000\000\000\000\000\000\000\000\000\000\n\000\000\000\000\000\000\000\200\322\326W\f\177\000\000\000\000\000\000\000\000\000\000\066\303\017X\f\177\000\000\000"...}
name = 0x7f0c3c87bef0
rdsiter = 0x0
rdataset = {magic = 1145983826, methods = 0x0, link = {prev = 0xffffffffffffffff, next = 0xffffffffffffffff}, rdclass = 0, type = 0, ttl = 0, trust = 0, covers = 0, attributes = 0, count = 4294967295, resign = 0, {keytable = {node = 0x0, iter = 0x0}, ncache = {raw = 0x0, iter_pos = 0x0, iter_count = 0}, slab = {db = 0x0, node = 0x0, raw = 0x0, iter_pos = 0x0, iter_count = 0, noqname = 0x0, closest = 0x0}, rdlist = {list = 0x0, iter = 0x0, noqname = 0x0, closest = 0x0, node = 0x0}, rps = {db = 0x0, iter_pos = 0x0, iter_count = 0}}}
bname = "catalog-tls.example", '\000' <repeats 413 times>...
cname = "\000\274\207<\f\177\000\000\001\000\000\000\001\000\000\000@y\":\f\177", '\000' <repeats 19 times>, "\204$:\f\177\000\000\000\340$:\f\177", '\000' <repeats 11 times>, "~`@\f\177", '\000' <repeats 18 times>, "\340\300\207<\f\177\000\000\200\250\340S\f\177\000\000\000\000\000\000\000\000\000\000 \200d@\f\177\000\000\020", '\000' <repeats 311 times>...
is_vers_processed = <optimized out>
is_active = true
vers = 2
catz_vers = <optimized out>
__func__ = "dns__catz_update_cb"
#9 0x00007f0c5b856976 in isc__work_cb (req=<optimized out>) at work.c:30
work = 0x7f0c53e6dbe0
#10 0x00007f0c592254ee in worker () from /lib64/libuv.so.1
No symbol table info available.
#11 0x00007f0c580f81da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#12 0x00007f0c573cbe73 in clone () from /lib64/libc.so.6
No symbol table info available.
```May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4400CID 467118: Control flow issue in lib/dns/message.c2024-02-24T07:55:29ZMichal NowakCID 467118: Control flow issue in lib/dns/message.cCoverity Scan claims control flow issue in `lib/dns/message.c` (suspect: !8400).
```
*** CID 467118: Control flow issues (DEADCODE)
/lib/dns/message.c: 1076 in getquestions()
1070 return (DNS_R_RECOVERABLE);
1071 }
1072 ...Coverity Scan claims control flow issue in `lib/dns/message.c` (suspect: !8400).
```
*** CID 467118: Control flow issues (DEADCODE)
/lib/dns/message.c: 1076 in getquestions()
1070 return (DNS_R_RECOVERABLE);
1071 }
1072 return (ISC_R_SUCCESS);
1073
1074 cleanup:
1075 if (rdataset != NULL) {
>>> CID 467118: Control flow issues (DEADCODE)
>>> Execution cannot reach this statement: "dns_message_puttemprdataset...".
1076 dns_message_puttemprdataset(msg, &rdataset);
1077 }
1078 if (free_name) {
1079 dns_message_puttempname(msg, &name);
1080 }
1081
```May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4543Re-enable unreachable checks in dnssec system test2024-02-24T07:55:26ZTom KrizekRe-enable unreachable checks in dnssec system testIn https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/8085, a premature [exit statement](https://gitlab.isc.org/isc-projects/bind9/-/blob/b54bdf8d78666d8dcc6d4e1ad74c4af0a130e1a8/bin/tests/system/dnssec/tests.sh#L3711) has been a...In https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/8085, a premature [exit statement](https://gitlab.isc.org/isc-projects/bind9/-/blob/b54bdf8d78666d8dcc6d4e1ad74c4af0a130e1a8/bin/tests/system/dnssec/tests.sh#L3711) has been accidentally added to the `dnssec` test, making the remaining checks unreachable.May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Tom KrizekTom Krizekhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4581CID 486476: Memory - corruptions (OVERRUN) in lib/dns/resconf.c2024-02-24T07:55:23ZMichal NowakCID 486476: Memory - corruptions (OVERRUN) in lib/dns/resconf.cAfter 371defc35753d04fa8b769b8c859630c3a76e9ed, Coverity Scan claims memory corruption in `lib/dns/resconf.c`:
```cpp
/lib/dns/resconf.c: 246 in add_server()
240
241 /* XXX: special case: treat all-0 IPv4 address as loopback *...After 371defc35753d04fa8b769b8c859630c3a76e9ed, Coverity Scan claims memory corruption in `lib/dns/resconf.c`:
```cpp
/lib/dns/resconf.c: 246 in add_server()
240
241 /* XXX: special case: treat all-0 IPv4 address as loopback */
242 v4 = &((struct sockaddr_in *)res->ai_addr)->sin_addr;
243 if (memcmp(v4, zeroaddress, 4) == 0) {
244 memmove(v4, loopaddress, 4);
245 }
>>> CID 486476: Memory - corruptions (OVERRUN)
>>> Overrunning struct type sockaddr_in of 16 bytes by passing it to a function which accesses it at byte offset 27 using argument "res->ai_addrlen" (which evaluates to 28). [Note: The source code implementation of the function has been overridden by a builtin model.]
246 memmove(&address->type.sin, res->ai_addr, res->ai_addrlen);
247 } else if (res->ai_family == AF_INET6) {
248 memmove(&address->type.sin6, res->ai_addr, res->ai_addrlen);
249 } else {
250 isc_mem_put(mctx, address, sizeof(*address));
251 UNEXPECTED_ERROR("ai_family (%d) not INET nor INET6",
```May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/3472IPv4-only mode not respected for zone transfers2024-02-24T07:55:11ZThomas AmgartenIPv4-only mode not respected for zone transfers### Summary
Running BIND in IPv4-only-mode (```named -4```) and using a mirror zone (local root), then BIND tries to AXFR the root zone with IPv6 and reports failures about the unreachability:
```
27-Jul-2022 08:57:43.309 general: info:...### Summary
Running BIND in IPv4-only-mode (```named -4```) and using a mirror zone (local root), then BIND tries to AXFR the root zone with IPv6 and reports failures about the unreachability:
```
27-Jul-2022 08:57:43.309 general: info: zone ./IN: refresh: failure trying primary 2001:500:2::c#53 (source ::#0): operation canceled
27-Jul-2022 08:57:43.309 general: info: zone ./IN: refresh: failure trying primary 2001:500:2f::f#53 (source ::#0): operation canceled
27-Jul-2022 08:57:43.809 general: info: zone ./IN: refresh: failure trying primary 2001:500:12::d0d#53 (source ::#0): operation canceled
27-Jul-2022 08:57:43.809 general: info: zone ./IN: refresh: failure trying primary 2001:7fd::1#53 (source ::#0): operation canceled
27-Jul-2022 08:57:44.309 general: info: zone ./IN: refresh: failure trying primary 2620:0:2830:202::132#53 (source ::#0): operation canceled
27-Jul-2022 08:57:44.309 general: info: zone ./IN: refresh: failure trying primary 2620:0:2d0:202::132#53 (source ::#0): operation canceled
```
### BIND version used
```
$ named -V
BIND 9.18.5 (Stable Release) <id:6593103>
running on Linux x86_64 4.18.0-305.10.2.el8_4.x86_64 #1 SMP Tue Jul 20 20:34:55 UTC 2021
built by make with '--prefix=/usr/local/bind-9.18.5' '--sysconfdir=/opt/chroot/bind/etc/named/' '--mandir=/usr/local/share/man' '--localstatedir=/opt/chroot/bind/var' '--enable-largefile' '--enable-full-report' '--without-gssapi' '--with-json-c' '--enable-singletrace' 'PKG_CONFIG_PATH=:/usr/local/libuv/lib/pkgconfig/'
compiled by GCC 8.4.1 20200928 (Red Hat 8.4.1-1)
compiled with OpenSSL version: OpenSSL 1.1.1g FIPS 21 Apr 2020
linked to OpenSSL version: OpenSSL 1.1.1g FIPS 21 Apr 2020
compiled with libuv version: 1.41.1
linked to libuv version: 1.41.1
compiled with libnghttp2 version: 1.33.0
linked to libnghttp2 version: 1.33.0
compiled with json-c version: 0.13.1
linked to json-c version: 0.13.1
compiled with zlib version: 1.2.11
linked to zlib version: 1.2.11
threads support is enabled
default paths:
named configuration: /opt/chroot/bind/etc/named/named.conf
rndc configuration: /opt/chroot/bind/etc/named/rndc.conf
DNSSEC root key: /opt/chroot/bind/etc/named/bind.keys
nsupdate session key: /opt/chroot/bind/var/run/named/session.key
named PID file: /opt/chroot/bind/var/run/named/named.pid
named lock file: /opt/chroot/bind/var/run/named/named.lock
```
### Steps to reproduce
Create a mirror zone:
```
zone "." {
type mirror;
notify no;
};
```
Run BIND with IPv4-only:
```
$ /usr/local/bind/sbin/named -4 -t /opt/chroot/bind -u named -c /etc/named/named.conf
```
And now check the log for the IPv6 failure:
```
27-Jul-2022 09:18:59.148 general: info: zone ./IN: refresh: failure trying primary 2001:500:200::b#53 (source ::#0): operation canceled
27-Jul-2022 09:18:59.651 general: info: zone ./IN: refresh: failure trying primary 2001:500:2::c#53 (source ::#0): operation canceled
27-Jul-2022 09:18:59.651 general: info: zone ./IN: refresh: failure trying primary 2001:500:2f::f#53 (source ::#0): operation canceled
27-Jul-2022 09:19:00.151 general: info: zone ./IN: refresh: failure trying primary 2001:500:12::d0d#53 (source ::#0): operation canceled
27-Jul-2022 09:19:00.151 general: info: zone ./IN: refresh: failure trying primary 2001:7fd::1#53 (source ::#0): operation canceled
27-Jul-2022 09:19:00.651 general: info: zone ./IN: refresh: failure trying primary 2620:0:2830:202::132#53 (source ::#0): operation canceled
27-Jul-2022 09:19:00.651 general: info: zone ./IN: refresh: failure trying primary 2620:0:2d0:202::132#53 (source ::#0): operation canceled
```
### What is the current *bug* behavior?
BIND tries to AXFR the root zone over IPv6, although ```named``` is configured to run in IPv4-only-mode.
### What is the expected *correct* behavior?
Not trying to AXFR the mirror zone over IPv6.
### Relevant configuration files
### Relevant logs and/or screenshots
Failure in the log:
```
27-Jul-2022 09:11:18.990 general: info: zone ./IN: refresh: failure trying primary 2001:500:2::c#53 (source ::#0): operation canceled
27-Jul-2022 09:11:18.990 general: info: zone ./IN: refresh: failure trying primary 2001:500:2f::f#53 (source ::#0): operation canceled
27-Jul-2022 09:11:19.490 general: info: zone ./IN: refresh: failure trying primary 2001:500:12::d0d#53 (source ::#0): operation canceled
27-Jul-2022 09:11:19.490 general: info: zone ./IN: refresh: failure trying primary 2001:7fd::1#53 (source ::#0): operation canceled
27-Jul-2022 09:11:19.990 general: info: zone ./IN: refresh: failure trying primary 2620:0:2830:202::132#53 (source ::#0): operation canceled
27-Jul-2022 09:11:19.990 general: info: zone ./IN: refresh: failure trying primary 2620:0:2d0:202::132#53 (source ::#0): operation canceled
```
### Possible fixesMay 2024 (9.18.27, 9.18.27-S1, 9.19.24)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/3987Change DNSKEY TTL of inline-signed zone2024-02-24T07:55:08ZGerald VogtChange DNSKEY TTL of inline-signed zone### Description
I have a few zones using inline-signing which I have set up originally with 2d TTL. Due to this the existing DNSKEY RRs also have 2d TTL. Now I have been trying to reduce the TTL to 1d but it seems there is no supported ...### Description
I have a few zones using inline-signing which I have set up originally with 2d TTL. Due to this the existing DNSKEY RRs also have 2d TTL. Now I have been trying to reduce the TTL to 1d but it seems there is no supported way or tool to do so.
I have set dnskey-ttl to 1d and replace keys, still all DNSKEY RRs have 2d TTL. Setting it on the key with dnssec-settime doesn't help either and man pages specifically mention:
```
This option sets the default TTL to use for this key when it is converted into a DNSKEY RR.
This is the TTL used when the key is imported into a zone, unless there was already a DNSKEY
RRset in place, in which case the existing TTL takes precedence.
```
Running AlmaLinux 9 bind-9.16.23-5.el9_1.x86_64.
### Request
Add some way to change the TTL of DNSKEY RRs in inline-signed zones.
### Links / references
I have found a thread from 2016 about the same problem: https://www.mail-archive.com/bind-users@lists.isc.org/msg23186.htmlMay 2024 (9.18.27, 9.18.27-S1, 9.19.24)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/4104ZoneQuota stats counter is not counting everything2024-02-24T07:55:05ZOndřej SurýZoneQuota stats counter is not counting everythingThe `ZoneQuota` should log all the hits to `fcount_incr()` returning `ISC_R_QUOTA`, but it does only in a single place. The counting should be moved to `fctx_incr()`.The `ZoneQuota` should log all the hits to `fcount_incr()` returning `ISC_R_QUOTA`, but it does only in a single place. The counting should be moved to `fctx_incr()`.May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4423named starts up slow when many zones reference the same dnssec-policy2024-02-24T07:54:22ZMatthijs Mekkingmatthijs@isc.orgnamed starts up slow when many zones reference the same dnssec-policyWhile rolling out KASP to many zones, it is more efficient to use more DNSSEC policies in order to improve
reload/reconfig times.
When all zones or referenced by the same `dnssec-policy`, it takes quite some time to process all zones af...While rolling out KASP to many zones, it is more efficient to use more DNSSEC policies in order to improve
reload/reconfig times.
When all zones or referenced by the same `dnssec-policy`, it takes quite some time to process all zones after reload/reconfig and CPU usage of the named process remains at 100% and it takes quite a few minutes for named to start responding to queries after such a reload/reconfig request.
When spreading my zones to 10 identical policies, cpu usage goes well above 100% (using more threads I assume) and this is speeding
things up really nice.May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/4453Switching to a different dnssec-policy broke my zone.2024-02-24T07:54:16ZBjörn PerssonSwitching to a different dnssec-policy broke my zone.### Summary
My zone was previously signed with a KSK and a ZSK with unlimited lifetime. I switched the zone over to a dnssec-policy using CSKs and automatic key rotation. After the DS record was updated, most of the RRSIG records were r...### Summary
My zone was previously signed with a KSK and a ZSK with unlimited lifetime. I switched the zone over to a dnssec-policy using CSKs and automatic key rotation. After the DS record was updated, most of the RRSIG records were removed, leaving the zone broken to validating resolvers.
### BIND version used
```
# named -V
BIND 9.18.19-1~deb12u1-Debian (Extended Support Version) <id:>
running on Linux x86_64 5.10.0-26-amd64 #1 SMP Debian 5.10.197-1 (2023-09-29)
built by make with '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/x86_64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--disable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=yes' '--with-libidn2' '--with-json-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' '--enable-dnstap' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -ffile-prefix-map=/build/reproducible-path/bind9-9.18.19=. -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
compiled by GCC 12.2.0
compiled with OpenSSL version: OpenSSL 3.0.10 1 Aug 2023
linked to OpenSSL version: OpenSSL 3.0.11 19 Sep 2023
compiled with libuv version: 1.44.2
linked to libuv version: 1.44.2
compiled with libnghttp2 version: 1.52.0
linked to libnghttp2 version: 1.52.0
compiled with libxml2 version: 2.9.14
linked to libxml2 version: 20914
compiled with json-c version: 0.16
linked to json-c version: 0.16
compiled with zlib version: 1.2.13
linked to zlib version: 1.2.13
linked to maxminddb version: 1.7.1
compiled with protobuf-c version: 1.4.1
linked to protobuf-c version: 1.4.1
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): yes
default paths:
named configuration: /etc/bind/named.conf
rndc configuration: /etc/bind/rndc.conf
DNSSEC root key: /etc/bind/bind.keys
nsupdate session key: //run/named/session.key
named PID file: //run/named/named.pid
named lock file: //run/named/named.lock
geoip-directory: /usr/share/GeoIP
```
### Steps to reproduce
I have two zones that both exist in an external and an internal view. Each zone was previously signed with a KSK and a ZSK with unlimited lifetime. To proceed cautiously with the change to `dnssec-policy` I defined one policy that matched the existing keys and another that would use CSKs and automatic key rotation:
```
dnssec-policy "as_it_was" {
keys {
ksk lifetime unlimited algorithm rsasha256 2048;
zsk lifetime unlimited algorithm rsasha256 2048;
};
dnskey-ttl P1D;
purge-keys 0;
};
dnssec-policy "automation" {
keys {
csk lifetime P1M algorithm rsasha256 2048;
};
dnskey-ttl P1D;
max-zone-ttl P1D;
signatures-validity P1W;
signatures-refresh P2D;
};
```
First I switched the zones from "`auto-dnssec maintain;`" to "`dnssec-policy as_it_was;`". Bind continued using the existing keys. Once I had the exchange of CDS and DS records working between my zones and the parent zone, I switched one zone over to "`dnssec-policy automation;`" in both views.
The rollover from the old keys to a CSK seemed to go smoothly, but after a while I discovered that the zone was only partially signed in the external view. Several records lacked RRSIG records. Dynamic updates of the unsigned records caused corresponding RRSIG records to appear.
After that initial problem, the following rollover from one CSK to another worked fine, so I proceeded to switch the second zone over to "`dnssec-policy automation;`". This time I took notes and watched for missing signatures.
2023-11-18 16:05:49 a CSK was generated. DNSKEY, CDS and CDNSKEY were signed with both the old KSK and the CSK. SOA got a new signature by the old ZSK. All other records kept their existing signatures.
2023-11-19 17:10:49 CDS and CDNSKEY records for the CSK were published. DNSKEY, CDS and CDNSKEY got new signatures by the KSK and the CSK. SOA was signed with the ZSK and the CSK.
2023-11-20 17:10:49 Bind noticed that DS had been updated in the parent zone.
2023-11-20 18:15:49 the ZSK and all its signatures were removed. DNSKEY, CDS and CDNSKEY got new signatures by the CSK and the KSK. SOA got a new signature by the CSK. All other records were left without RRSIG records.
This time I fixed the external view with "`rndc sign xn--rombobjrn-67a.se IN external`". All the unsigned records were then signed with the CSK. DNSKEY, CDS, CDNSKEY and SOA had their signatures renewed. I left the internal view alone.
2023-11-21 19:10:50 the KSK was removed. DNSKEY, CDS, CDNSKEY and SOA got new signatures by the CSK. At the same time, many but not all other records in the internal view were finally signed with the CSK, having lacked signatures for 24 hours and 55 minutes. Some more records were signed a few minutes later.
As I'm posting this, one NS and one MX record in the internal view are still unsigned after more than four days.
### What is the current *bug* behavior?
The zone becomes only partially signed. Validating resolvers reject the unsigned records.
### What is the expected *correct* behavior?
All records should be signed with the new key before the old keys and signatures are removed.
### Relevant configuration files
See the policies above. After the changes, all the zone declarations look essentially like this:
```
zone "xn--rombobjrn-67a.se" {
type master;
file "/var/lib/bind/db.xn--rombobjrn-67a.se.external";
dnssec-policy automation;
parental-agents { ::1; };
inline-signing no;
update-policy { [omitted] };
};
```
### Relevant logs and/or screenshots
Excerpts from the system log:
```
2023-11-19T17:10:49.436468+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: reconfiguring zone keys
2023-11-19T17:10:49.437286+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: reconfiguring zone keys
2023-11-19T17:10:49.488666+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-19T17:10:49.489192+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-19T17:10:49.501444+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now inactive
2023-11-19T17:10:49.502076+01:00 cutie named[443161]: CDS (SHA-256) for key xn--rombobjrn-67a.se/RSASHA256/53584 is now deleted
2023-11-19T17:10:49.502515+01:00 cutie named[443161]: CDNSKEY for key xn--rombobjrn-67a.se/RSASHA256/53584 is now deleted
2023-11-19T17:10:49.502904+01:00 cutie named[443161]: CDS for key xn--rombobjrn-67a.se/RSASHA256/17339 is now published
2023-11-19T17:10:49.503279+01:00 cutie named[443161]: CDNSKEY for key xn--rombobjrn-67a.se/RSASHA256/17339 is now published
2023-11-19T17:10:49.530343+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-19T17:10:49.530897+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-19T17:10:49.534298+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now inactive
2023-11-19T17:10:49.534962+01:00 cutie named[443161]: CDS (SHA-256) for key xn--rombobjrn-67a.se/RSASHA256/53584 is now deleted
2023-11-19T17:10:49.535337+01:00 cutie named[443161]: CDNSKEY for key xn--rombobjrn-67a.se/RSASHA256/53584 is now deleted
2023-11-19T17:10:49.535684+01:00 cutie named[443161]: CDS for key xn--rombobjrn-67a.se/RSASHA256/17339 is now published
2023-11-19T17:10:49.536038+01:00 cutie named[443161]: CDNSKEY for key xn--rombobjrn-67a.se/RSASHA256/17339 is now published
2023-11-19T17:10:49.637732+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: next key event: 19-Nov-2023 18:10:49.432
2023-11-19T17:10:49.638433+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: sending notifies (serial 2023092737)
2023-11-19T17:10:49.651717+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: next key event: 19-Nov-2023 18:10:49.432
2023-11-19T17:10:49.673263+01:00 cutie named[443161]: client @0x7efdf9b21368 10.1.0.5#54619 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR started (serial 2023092736 -> 2023092737)
2023-11-19T17:10:49.674244+01:00 cutie named[443161]: client @0x7efdf9b21368 10.1.0.5#54619 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR ended: 1 messages, 23 records, 5465 bytes, 0.004 secs (1366250 bytes/sec) (serial 2023092737)
2023-11-19T17:10:50.192637+01:00 cutie named[443161]: client @0x7efdfa51af68 10.1.2.1#57043 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR started (serial 2023092736 -> 2023092737)
2023-11-19T17:10:50.193661+01:00 cutie named[443161]: client @0x7efdfa51af68 10.1.2.1#57043 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR ended: 1 messages, 23 records, 5465 bytes, 0.001 secs (5465000 bytes/sec) (serial 2023092737)
```
```
2023-11-20T17:10:49.472806+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: reconfiguring zone keys
2023-11-20T17:10:49.473891+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: reconfiguring zone keys
2023-11-20T17:10:49.525113+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-20T17:10:49.525655+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-20T17:10:49.529210+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now inactive
2023-11-20T17:10:49.530341+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: next key event: 20-Nov-2023 18:10:49.466
2023-11-20T17:10:49.557565+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-20T17:10:49.558183+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-20T17:10:49.561418+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now inactive
2023-11-20T17:10:49.562620+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: next key event: 20-Nov-2023 18:10:49.466
2023-11-20T17:10:49.617384+01:00 cutie named[443161]: keymgr: checkds DS for key xn--rombobjrn-67a.se/RSASHA256/17339 seen published at Mon Nov 20 17:10:49 2023
2023-11-20T17:10:49.621343+01:00 cutie named[443161]: keymgr: checkds DS for key xn--rombobjrn-67a.se/RSASHA256/53584 seen withdrawn at Mon Nov 20 17:10:49 2023
2023-11-20T17:10:49.624985+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: reconfiguring zone keys
2023-11-20T17:10:49.667546+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-20T17:10:49.668097+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-20T17:10:49.671602+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now inactive
2023-11-20T17:10:49.672714+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: next key event: 20-Nov-2023 18:15:49.618
2023-11-20T17:10:50.027333+01:00 cutie named[443161]: keymgr: checkds DS for key xn--rombobjrn-67a.se/RSASHA256/17339 seen published at Mon Nov 20 17:10:50 2023
2023-11-20T17:10:50.031352+01:00 cutie named[443161]: keymgr: checkds DS for key xn--rombobjrn-67a.se/RSASHA256/53584 seen withdrawn at Mon Nov 20 17:10:50 2023
2023-11-20T17:10:50.035151+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: reconfiguring zone keys
2023-11-20T17:10:50.077904+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-20T17:10:50.078540+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-20T17:10:50.081828+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now inactive
2023-11-20T17:10:50.083015+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: next key event: 20-Nov-2023 18:15:49.030
```
```
2023-11-20T18:15:49.036472+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: reconfiguring zone keys
2023-11-20T18:15:49.076389+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-20T18:15:49.077010+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-20T18:15:49.088905+01:00 cutie named[443161]: Removing expired key xn--rombobjrn-67a.se/13398/RSASHA256 from DNSKEY RRset.
2023-11-20T18:15:49.089406+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK) is now deleted
2023-11-20T18:15:49.089784+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now inactive
2023-11-20T18:15:49.192756+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: next key event: 20-Nov-2023 18:20:49.033
2023-11-20T18:15:49.193416+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: sending notifies (serial 2023092738)
2023-11-20T18:15:49.275467+01:00 cutie named[443161]: client @0x7efdebdc6d68 10.1.0.5#41397 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR started (serial 2023092737 -> 2023092739)
2023-11-20T18:15:49.278365+01:00 cutie named[443161]: client @0x7efdebdc6d68 10.1.0.5#41397 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR ended: 3 messages, 128 records, 38648 bytes, 0.004 secs (9662000 bytes/sec) (serial 2023092739)
2023-11-20T18:15:49.622949+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: reconfiguring zone keys
2023-11-20T18:15:49.664238+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-20T18:15:49.664712+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-20T18:15:49.667624+01:00 cutie named[443161]: Removing expired key xn--rombobjrn-67a.se/13398/RSASHA256 from DNSKEY RRset.
2023-11-20T18:15:49.668019+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK) is now deleted
2023-11-20T18:15:49.668373+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now inactive
2023-11-20T18:15:49.764336+01:00 cutie named[443161]: client @0x7efdebdc5168 10.1.2.1#58091 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR started (serial 2023092737 -> 2023092739)
2023-11-20T18:15:49.767341+01:00 cutie named[443161]: client @0x7efdebdc5168 10.1.2.1#58091 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR ended: 3 messages, 128 records, 38648 bytes, 0.004 secs (9662000 bytes/sec) (serial 2023092739)
2023-11-20T18:15:49.779256+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: next key event: 20-Nov-2023 18:20:49.621
2023-11-20T18:15:54.192437+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: sending notifies (serial 2023092739)
```
```
2023-11-21T13:15:40.402451+01:00 cutie named[443161]: received control channel command 'sign xn--rombobjrn-67a.se IN external'
2023-11-21T13:15:40.405362+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: reconfiguring zone keys
2023-11-21T13:15:40.431241+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-21T13:15:40.431697+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-21T13:15:40.433742+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now inactive
2023-11-21T13:15:40.528574+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: next key event: 21-Nov-2023 19:10:50.395
2023-11-21T13:15:40.529172+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: sending notifies (serial 2023092740)
2023-11-21T13:15:40.773096+01:00 cutie named[443161]: client @0x7efdfa51af68 10.1.0.5#33623 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR started (serial 2023092739 -> 2023092742)
2023-11-21T13:15:40.774513+01:00 cutie named[443161]: client @0x7efdfa51af68 10.1.0.5#33623 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR ended: 1 messages, 46 records, 12419 bytes, 0.004 secs (3104750 bytes/sec) (serial 2023092742)
2023-11-21T13:15:41.172719+01:00 cutie named[443161]: client @0x7efdf9b20568 10.1.2.1#33203 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR started (serial 2023092739 -> 2023092745)
2023-11-21T13:15:41.174657+01:00 cutie named[443161]: client @0x7efdf9b20568 10.1.2.1#33203 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR ended: 2 messages, 89 records, 24907 bytes, 0.004 secs (6226750 bytes/sec) (serial 2023092745)
2023-11-21T13:15:45.528370+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: sending notifies (serial 2023092750)
2023-11-21T13:15:45.561710+01:00 cutie named[443161]: client @0x7efdebdc6d68 10.1.0.5#52787 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR started (serial 2023092742 -> 2023092750)
2023-11-21T13:15:45.564494+01:00 cutie named[443161]: client @0x7efdebdc6d68 10.1.0.5#52787 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR ended: 2 messages, 114 records, 31108 bytes, 0.004 secs (7777000 bytes/sec) (serial 2023092750)
2023-11-21T13:15:46.078928+01:00 cutie named[443161]: client @0x7efdfa51bd68 10.1.2.1#60701 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR started (serial 2023092745 -> 2023092750)
2023-11-21T13:15:46.080874+01:00 cutie named[443161]: client @0x7efdfa51bd68 10.1.2.1#60701 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR ended: 2 messages, 71 records, 18769 bytes, 0.001 secs (18769000 bytes/sec) (serial 2023092750)
```
```
2023-11-21T19:10:50.400377+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: reconfiguring zone keys
2023-11-21T19:10:50.432532+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-21T19:10:50.433038+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-21T19:10:50.443664+01:00 cutie named[443161]: Removing expired key xn--rombobjrn-67a.se/53584/RSASHA256 from DNSKEY RRset.
2023-11-21T19:10:50.444123+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now deleted
2023-11-21T19:10:50.511795+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: next key event: 21-Nov-2023 19:15:50.396
2023-11-21T19:10:50.512265+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: sending notifies (serial 2023092751)
2023-11-21T19:10:50.576696+01:00 cutie named[443161]: client @0x7efdfa51af68 10.1.0.5#54307 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR started (serial 2023092750 -> 2023092752)
2023-11-21T19:10:50.577645+01:00 cutie named[443161]: client @0x7efdfa51af68 10.1.0.5#54307 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR ended: 1 messages, 27 records, 5832 bytes, 0.001 secs (5832000 bytes/sec) (serial 2023092752)
2023-11-21T19:10:50.626991+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: reconfiguring zone keys
2023-11-21T19:10:50.660686+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-21T19:10:50.661150+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-21T19:10:50.663077+01:00 cutie named[443161]: Removing expired key xn--rombobjrn-67a.se/53584/RSASHA256 from DNSKEY RRset.
2023-11-21T19:10:50.663489+01:00 cutie named[443161]: DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK) is now deleted
2023-11-21T19:10:50.738310+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: next key event: 21-Nov-2023 19:15:50.624
2023-11-21T19:10:51.191122+01:00 cutie named[443161]: client @0x7efdf9b20568 10.1.2.1#43631 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR started (serial 2023092750 -> 2023092752)
2023-11-21T19:10:51.191859+01:00 cutie named[443161]: client @0x7efdf9b20568 10.1.2.1#43631 (xn--rombobjrn-67a.se): view external: transfer of 'xn--rombobjrn-67a.se/IN': IXFR ended: 1 messages, 27 records, 5832 bytes, 0.001 secs (5832000 bytes/sec) (serial 2023092752)
2023-11-21T19:10:55.511787+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: sending notifies (serial 2023092752)
2023-11-21T19:15:50.404325+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: reconfiguring zone keys
2023-11-21T19:15:50.427941+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-21T19:15:50.428397+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-21T19:15:50.440377+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: next key event: 21-Nov-2023 19:20:49.398
2023-11-21T19:15:50.630905+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: reconfiguring zone keys
2023-11-21T19:15:50.656580+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-21T19:15:50.657098+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-21T19:15:50.659929+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: next key event: 21-Nov-2023 19:20:49.626
2023-11-21T19:20:49.405293+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: reconfiguring zone keys
2023-11-21T19:20:49.429191+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-21T19:20:49.429646+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-21T19:20:49.438021+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: next key event: 21-Nov-2023 19:20:50.399
2023-11-21T19:20:49.630959+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: reconfiguring zone keys
2023-11-21T19:20:49.656677+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-21T19:20:49.657172+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-21T19:20:49.659897+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: next key event: 21-Nov-2023 19:20:50.627
2023-11-21T19:20:50.401138+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: reconfiguring zone keys
2023-11-21T19:20:50.427552+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-21T19:20:50.428010+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-21T19:20:50.434902+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/external: next key event: 21-Nov-2023 20:20:50.399
2023-11-21T19:20:50.629148+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: reconfiguring zone keys
2023-11-21T19:20:50.654607+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/13398 (ZSK)
2023-11-21T19:20:50.655054+01:00 cutie named[443161]: keymgr: retire DNSKEY xn--rombobjrn-67a.se/RSASHA256/53584 (KSK)
2023-11-21T19:20:50.657686+01:00 cutie named[443161]: zone xn--rombobjrn-67a.se/IN/internal: next key event: 21-Nov-2023 20:20:50.627
```
Some possibly useful status data from when the zone lacked signatures:
```
# rndc dnssec -status xn--rombobjrn-67a.se IN external
dnssec-policy: automatik
current time: Tue Nov 21 12:57:26 2023
key: 17339 (RSASHA256), CSK
published: yes - since Sat Nov 18 16:05:49 2023
key signing: yes - since Sat Nov 18 16:05:49 2023
zone signing: yes - since Sat Nov 18 16:05:49 2023
Next rollover scheduled on Mon Dec 18 15:00:49 2023
- goal: omnipresent
- dnskey: omnipresent
- ds: rumoured
- zone rrsig: omnipresent
- key rrsig: omnipresent
key: 13398 (RSASHA256), ZSK
published: no
zone signing: no
Key has been removed from the zone
- goal: hidden
- dnskey: hidden
- zone rrsig: unretentive
key: 53584 (RSASHA256), KSK
published: yes - since Sun Nov 3 04:26:07 2019
key signing: yes - since Sun Nov 3 04:26:07 2019
Rollover is due since Sun Nov 19 18:05:49 2023
- goal: hidden
- dnskey: omnipresent
- ds: unretentive
- key rrsig: omnipresent
# rndc zonestatus xn--rombobjrn-67a.se IN external
name: xn--rombobjrn-67a.se
type: primary
files: /var/lib/bind/db.xn--rombobjrn-67a.se.external
serial: 2023092739
nodes: 42
last loaded: Tue, 24 Oct 2023 12:43:57 GMT
secure: no
key maintenance: automatic
next key event: Tue, 21 Nov 2023 18:10:50 GMT
dynamic: yes
frozen: no
reconfigurable via modzone: no
```
The output of `rndc zonestatus` changed when I ran `rndc sign`:
```
# rndc zonestatus xn--rombobjrn-67a.se IN external
name: xn--rombobjrn-67a.se
type: primary
files: /var/lib/bind/db.xn--rombobjrn-67a.se.external
serial: 2023092750
nodes: 42
last loaded: Tue, 24 Oct 2023 12:43:57 GMT
secure: yes
inline signing: no
key maintenance: automatic
next key event: Tue, 21 Nov 2023 18:10:50 GMT
next resign node: 7c2ecd07f155648431e0f94b89247d713c5786e1e73e953f2fe7eca3._openpgpkey.xn--rombobjrn-67a.se/NSEC
next resign time: Wed, 22 Nov 2023 22:55:09 GMT
dynamic: yes
frozen: no
reconfigurable via modzone: no
```May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/4475Data races in isc_buffer_peekuint8, rdataset_settrust, and memmove2024-02-24T07:54:00ZMichal NowakData races in isc_buffer_peekuint8, rdataset_settrust, and memmoveJob [#3848477](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3848477) failed for c4fcdbefc5ac65e62f8d16ba78737aa6174c9592.
There are three new types of TSAN errors in the failed `respdiff-long:tsan` CI job.
I did not happen [yesterd...Job [#3848477](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3848477) failed for c4fcdbefc5ac65e62f8d16ba78737aa6174c9592.
There are three new types of TSAN errors in the failed `respdiff-long:tsan` CI job.
I did not happen [yesterday](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3843810) on 64ef6968f379fa220c2a2d76311705b4e248e286, so should this be caused by a new code, the only theoretically relevant MR is !8515.
```
WARNING: ThreadSanitizer: data race
Read of size 1 at 0x000000000001 by main thread:
#0 isc_buffer_peekuint8 ../../lib/isc/include/isc/buffer.h:847
#1 isc_buffer_getuint8 ../../lib/isc/include/isc/buffer.h:854
#2 dns_ncache_getsigrdataset lib/dns/ncache.c:630
#3 validate_ncache lib/dns/validator.c:2388
#4 validate_nx lib/dns/validator.c:2431
#5 validator_start lib/dns/validator.c:2994
#6 isc__async_cb lib/isc/async.c:111
#7 uv__async_io /usr/src/libuv-v1.47.0/src/unix/async.c:176
#8 thread_body lib/isc/thread.c:85
#9 isc_thread_main lib/isc/thread.c:116
#10 isc_loopmgr_run lib/isc/loop.c:454
#11 main bin/named/main.c:1574
Previous write of size 1 at 0x000000000001 by thread T0001:
#0 rdataset_settrust lib/dns/ncache.c:499
#1 dns_rdataset_settrust lib/dns/rdataset.c:597
#2 marksecure lib/dns/validator.c:202
#3 validate_answer lib/dns/validator.c:1528
#4 validator_start lib/dns/validator.c:2935
#5 isc__async_cb lib/isc/async.c:111
#6 uv__async_io /usr/src/libuv-v1.47.0/src/unix/async.c:176
#7 thread_body lib/isc/thread.c:85
#8 thread_run lib/isc/thread.c:100
Location is heap block of size 1015 at 0x000000000020 allocated by main thread:
#0 malloc ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:647
#1 mallocx lib/isc/jemalloc_shim.h:67
#2 mem_get lib/isc/mem.c:303
#3 isc__mem_get lib/isc/mem.c:675
#4 dns_rdataslab_fromrdataset lib/dns/rdataslab.c:332
#5 dns__rbtdb_addrdataset lib/dns/rbtdb.c:3153
#6 dns__db_addrdataset lib/dns/db.c:681
#7 addoptout lib/dns/ncache.c:283
#8 dns_ncache_add lib/dns/ncache.c:103
#9 ncache_adderesult lib/dns/resolver.c:6358
#10 validated lib/dns/resolver.c:5385
#11 validator_done_cb lib/dns/validator.c:210
#12 isc__async_cb lib/isc/async.c:111
#13 uv__async_io /usr/src/libuv-v1.47.0/src/unix/async.c:176
#14 thread_body lib/isc/thread.c:85
#15 isc_thread_main lib/isc/thread.c:116
#16 isc_loopmgr_run lib/isc/loop.c:454
#17 main bin/named/main.c:1574
Thread T0001 'isc-loop-0002' (running) created by main thread at:
#0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1001
#1 isc_thread_create lib/isc/thread.c:139
#2 isc_loopmgr_run lib/isc/loop.c:448
#3 main bin/named/main.c:1574
SUMMARY: ThreadSanitizer: data race ../../lib/isc/include/isc/buffer.h:847 in isc_buffer_peekuint8
```
```
WARNING: ThreadSanitizer: data race
Write of size 1 at 0x000000000001 by main thread:
#0 rdataset_settrust lib/dns/ncache.c:499
#1 dns_rdataset_settrust lib/dns/rdataset.c:597
#2 marksecure lib/dns/validator.c:202
#3 validate_answer lib/dns/validator.c:1528
#4 validator_start lib/dns/validator.c:2935
#5 isc__async_cb lib/isc/async.c:111
#6 uv__async_io /usr/src/libuv-v1.47.0/src/unix/async.c:176
#7 thread_body lib/isc/thread.c:85
#8 isc_thread_main lib/isc/thread.c:116
#9 isc_loopmgr_run lib/isc/loop.c:454
#10 main bin/named/main.c:1574
Previous write of size 1 at 0x000000000001 by thread T0001:
#0 rdataset_settrust lib/dns/ncache.c:499
#1 dns_rdataset_settrust lib/dns/rdataset.c:597
#2 marksecure lib/dns/validator.c:202
#3 validate_answer lib/dns/validator.c:1528
#4 validator_start lib/dns/validator.c:2935
#5 isc__async_cb lib/isc/async.c:111
#6 uv__async_io /usr/src/libuv-v1.47.0/src/unix/async.c:176
#7 thread_body lib/isc/thread.c:85
#8 thread_run lib/isc/thread.c:100
Location is heap block of size 1015 at 0x000000000014 allocated by thread T0002:
#0 malloc ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:647
#1 mallocx lib/isc/jemalloc_shim.h:67
#2 mem_get lib/isc/mem.c:303
#3 isc__mem_get lib/isc/mem.c:675
#4 dns_rdataslab_fromrdataset lib/dns/rdataslab.c:332
#5 dns__rbtdb_addrdataset lib/dns/rbtdb.c:3153
#6 dns__db_addrdataset lib/dns/db.c:681
#7 addoptout lib/dns/ncache.c:283
#8 dns_ncache_add lib/dns/ncache.c:103
#9 ncache_adderesult lib/dns/resolver.c:6358
#10 validated lib/dns/resolver.c:5385
#11 validator_done_cb lib/dns/validator.c:210
#12 isc__async_cb lib/isc/async.c:111
#13 uv__async_io /usr/src/libuv-v1.47.0/src/unix/async.c:176
#14 thread_body lib/isc/thread.c:85
#15 thread_run lib/isc/thread.c:100
Thread T0001 'isc-loop-0001' (running) created by main thread at:
#0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1001
#1 isc_thread_create lib/isc/thread.c:139
#2 isc_loopmgr_run lib/isc/loop.c:448
#3 main bin/named/main.c:1574
Thread T0002 'isc-loop-0002' (running) created by main thread at:
#0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1001
#1 isc_thread_create lib/isc/thread.c:139
#2 isc_loopmgr_run lib/isc/loop.c:448
#3 main bin/named/main.c:1574
SUMMARY: ThreadSanitizer: data race lib/dns/ncache.c:499 in rdataset_settrust
```
```
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by main thread:
#0 memmove ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:810
#1 memmove ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:808
#2 memmove /usr/include/x86_64-linux-gnu/bits/string_fortified.h:36
#3 dns_name_fromregion lib/dns/name.c:739
#4 dns_ncache_current lib/dns/ncache.c:701
#5 validate_ncache lib/dns/validator.c:2382
#6 validate_nx lib/dns/validator.c:2431
#7 validator_start lib/dns/validator.c:2994
#8 isc__async_cb lib/isc/async.c:111
#9 uv__async_io /usr/src/libuv-v1.47.0/src/unix/async.c:176
#10 thread_body lib/isc/thread.c:85
#11 isc_thread_main lib/isc/thread.c:116
#12 isc_loopmgr_run lib/isc/loop.c:454
#13 main bin/named/main.c:1574
Previous write of size 1 at 0x000000000014 by thread T0001:
#0 rdataset_settrust lib/dns/ncache.c:499
#1 dns_rdataset_settrust lib/dns/rdataset.c:597
#2 marksecure lib/dns/validator.c:200
#3 validate_answer lib/dns/validator.c:1528
#4 validator_start lib/dns/validator.c:2935
#5 isc__async_cb lib/isc/async.c:111
#6 uv__async_io /usr/src/libuv-v1.47.0/src/unix/async.c:176
#7 thread_body lib/isc/thread.c:85
#8 thread_run lib/isc/thread.c:100
Location is heap block of size 1047 at 0x000000000021 allocated by thread T0001:
#0 malloc ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:647
#1 mallocx lib/isc/jemalloc_shim.h:67
#2 mem_get lib/isc/mem.c:303
#3 isc__mem_get lib/isc/mem.c:675
#4 dns_rdataslab_fromrdataset lib/dns/rdataslab.c:332
#5 dns__rbtdb_addrdataset lib/dns/rbtdb.c:3153
#6 dns__db_addrdataset lib/dns/db.c:681
#7 addoptout lib/dns/ncache.c:283
#8 dns_ncache_add lib/dns/ncache.c:103
#9 ncache_adderesult lib/dns/resolver.c:6358
#10 validated lib/dns/resolver.c:5385
#11 validator_done_cb lib/dns/validator.c:210
#12 isc__async_cb lib/isc/async.c:111
#13 uv__async_io /usr/src/libuv-v1.47.0/src/unix/async.c:176
#14 thread_body lib/isc/thread.c:85
#15 thread_run lib/isc/thread.c:100
Thread T0001 'isc-loop-0001' (running) created by main thread at:
#0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1001
#1 isc_thread_create lib/isc/thread.c:139
#2 isc_loopmgr_run lib/isc/loop.c:448
#3 main bin/named/main.c:1574
SUMMARY: ThreadSanitizer: data race /usr/include/x86_64-linux-gnu/bits/string_fortified.h:36 in memmove
```
I restarted the job, and this is a [reproducible issue](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3849392).May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/4517dnssec-verify reports errors in NSEC3 chain2024-02-24T07:53:57ZLibor Peltandnssec-verify reports errors in NSEC3 chain### Summary
Please see the attached zone file. The output of dnssec-verify is:
```
$ faketime '2023-12-10' dnssec-verify -o 6DA7ffbF. 6DA7ffbF.rndzone
Loading zone '6DA7ffbF.' from file '6DA7ffbF.rndzone'
Verifying the zone using the f...### Summary
Please see the attached zone file. The output of dnssec-verify is:
```
$ faketime '2023-12-10' dnssec-verify -o 6DA7ffbF. 6DA7ffbF.rndzone
Loading zone '6DA7ffbF.' from file '6DA7ffbF.rndzone'
Verifying the zone using the following algorithms:
- ECDSAP256SHA256
Bad NSEC3 record for fadb1aa3f.6DA7ffbF, bit map mismatch
Expected and found NSEC3 chains not equal
Break in NSEC3 chain at: VKGD3TE5QRGB6S0KJH6UV3FKS9FUMRIV
Expected: 01EAMK8ES71TN6TKHOK512LQMCORC5O9
Found: 0R6S95GSLHH7HT7MFN2N1NJGNFS7Q2CQ
DNSSEC completeness test failed (failure).
```
I'd say that the NSEC3 chain is however correct.
Some notes:
- opt-out is not used
- `fadb1aa3f.6da7ffbf.` -> `01eamk8es71tn6tkhok512lqmcorc5o9.6da7ffbf.` (first NSEC3 lexicographically, but this probably doesnt care)
- `427e09.owa.6da7ffbf.` -> `vkgd3te5qrgb6s0kjh6uv3fks9fumriv.6da7ffbf.` (last NSEC3 lexicographically)
- node `fadb1aa3f.6da7ffbf.` is "weird" in the way that it's a delegation with non-authoritative data: MX and even DNSKEY(!), but this shouldn't influence the chaining of NSEC3, moreover, it relates to the bitmap at 01EAMK... and not VKGD3T...
### BIND version affected
```
$ dnssec-verify -V
dnssec-verify 9.18.18-0ubuntu0.22.04.1-Ubuntu
```
### Steps to reproduce
Use faketime as the RRSIGs are expired already. It doesn't matter since the errors are related to NSEC3s and not signatures.
The zone file in question is attached.
Just call `$ faketime '2023-12-10' dnssec-verify -o 6DA7ffbF. 6DA7ffbF.rndzone`
### What is the current *bug* behavior?
Verify reports errors in the attached zone's NSEC3 chain.
### What is the expected *correct* behavior?
No errors reported.May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/4554Signature expiration calculation backwards compatibility bug2024-02-24T07:53:48ZMatthijs Mekkingmatthijs@isc.orgSignature expiration calculation backwards compatibility bugThe `signatures-refresh` option determines when RRSIG records need to be refreshed. Signatures that expire within this time are refreshed.
However, the code is also using this to determine the jitter. It uses a jitter range of 0 to `sig...The `signatures-refresh` option determines when RRSIG records need to be refreshed. Signatures that expire within this time are refreshed.
However, the code is also using this to determine the jitter. It uses a jitter range of 0 to `signatures-validity - signatures-refresh`) which is wrong: it should be using a range of 0 to `signatures-refresh`.
The `sig-validity-interval` that was used for `auto-dnssec` defined two parameters, the first being the signatures validity (same as `dnssec-policy`'s `signatures-validity`), the optional second one being the minimum bound of the signatures validity. It also serves as a signatures refresh. Basically the refresh value is the difference between the first and second parameter.
So the second parameter actually has two meanings: It serves as a jitter and a refresh value.
With `dnssec-policy` there is not yet a way to define `jitter`. The `signatures-refresh` is actually defined as the.
Two things need to be done:
- [x] Add a configuration option to `dnssec-policy` to set desired jitter.
- [x] Ensure resign interval is used correctly.May 2024 (9.18.27, 9.18.27-S1, 9.19.24)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/4579Restore the ability to select individual unit tests and turn on debugging2024-02-24T07:53:44ZMark AndrewsRestore the ability to select individual unit tests and turn on debugging63fe9312ff8f removed the ability to select individual tests from the command line and turn on debugging. This is useful when you want to check only parts of a unit test when developing. This restores / adds this ability.63fe9312ff8f removed the ability to select individual tests from the command line and turn on debugging. This is useful when you want to check only parts of a unit test when developing. This restores / adds this ability.May 2024 (9.18.27, 9.18.27-S1, 9.19.24)https://gitlab.isc.org/isc-projects/bind9/-/issues/3792incoming AXFR sometimes does not close TCP connection2024-02-24T07:53:11ZPetr Špačekpspacek@isc.orgincoming AXFR sometimes does not close TCP connection### Summary
I've noticed in PCAPs that sometimes BIND does not close TCP connection after successful incoming AXFR. This might cause source port depletion on a busy server.
### BIND version used
* ~"Affects v9.19": 9.19.9 56d7e01
* No...### Summary
I've noticed in PCAPs that sometimes BIND does not close TCP connection after successful incoming AXFR. This might cause source port depletion on a busy server.
### BIND version used
* ~"Affects v9.19": 9.19.9 56d7e01
* Not reproducible on ~"v9.18" (9.18.11 equivalent, b04ab06) - albeit closing the connection can take more than one second, it happens from the secondary side as expected
* ~"Affects v9.16": (9.16.37, b4a65aaea19762a3712932aa2270e8a833fbde22) - reproducible
Don't ask me how is that possible ...
### Steps to reproduce
1. Configure primary with 100k zones + catalog - can be BIND or Knot DNS (recommended to take BIND out of equation on one side)
2. Configure BIND as secondary for the catalog
3. Start secondary with clean state
### What is the current *bug* behavior?
PCAPs show that sometimes the primary closes hanging connection after primary-side timeout.
### What is the expected *correct* behavior?
Connections are closed as soon as possible.
### Relevant configuration files
#### Primary
* [named.conf](/uploads/863bf85788384d2e4893ea94cc606c89/named.conf)
* [catalog.db](/uploads/c515216922d648acf6065f7a50b36233/catalog.db)
* [empty.db](/uploads/5686c122ffb6fd4eb035bc1b88931e0f/empty.db)
Knot DNS version: [knotd.conf](/uploads/e59561f0b1f2047d348a51303d5a2119/knotd.conf)
#### Secondary
* [named.conf](/uploads/984a16e8322400cc6465b14ca45710ef/named.conf)
### Relevant logs and/or screenshots
* Primary: [primary.log.zst](/uploads/e50efe9e008cd762b3a671245e207b7d/primary.log.zst)
* Secondary: [secondary-for-knotd-conf3000.log.zst](/uploads/e8732553815f19ebf3b483629afd6279/secondary-for-knotd-conf3000.log.zst)
* search for `z19823.test` and look at timestamps
* PCAP: [bindconf3000.pcap.zst](/uploads/86b89c9ddc6d4e1c6066cfd1a997c25b/bindconf3000.pcap.zst)
* search for `tcp.stream eq 37322` in Wireshark to get `z19823.test` transfer
Suspicious conversation from the PCAP, times relative to the previous packet:
|No. | Time | Source | Source Port | Destination | Reply code | Info|
|--- | --- | --- | --- | --- | --- | ---|
|484345 | 0 | 192.0.2.2 | 40571 | 192.0.2.1 | | 40571 → 53 [SYN] Seq=0 Win=64660 Len=0 MSS=1220 SACK_PERM TSval=3661096036 TSecr=0 WS=128|
|484346 | 0,000027 | 192.0.2.1 | 53 | 192.0.2.2 | | 53 → 40571 [SYN, ACK] Seq=0 Ack=1 Win=65232 Len=0 MSS=1220 SACK_PERM TSval=1123290483 TSecr=3661096036 WS=128|
|484347 | 0,000008 | 192.0.2.2 | 40571 | 192.0.2.1 | | 40571 → 53 [ACK] Seq=1 Ack=1 Win=64768 Len=0 TSval=3661096036 TSecr=1123290483|
|511718 | 1,98078 | 192.0.2.2 | 40571 | 192.0.2.1 | | Standard query 0x47aa AXFR z19823.test|
|511719 | 0,000019 | 192.0.2.1 | 53 | 192.0.2.2 | | 53 → 40571 [ACK] Seq=1 Ack=32 Win=65280 Len=0 TSval=1123292464 TSecr=3661098017|
|511724 | 0,000107 | 192.0.2.1 | 53 | 192.0.2.2 | No error | Standard query response 0x47aa AXFR z19823.test SOA <Root> NS invalid SOA <Root>|
|511726 | 0,000009 | 192.0.2.2 | 40571 | 192.0.2.1 | | 40571 → 53 [ACK] Seq=32 Ack=121 Win=64768 Len=0 TSval=3661098017 TSecr=1123292464|
|601979 | 9,49634 | 192.0.2.1 | 53 | 192.0.2.2 | | 53 → 40571 [FIN, ACK] Seq=121 Ack=32 Win=65280 Len=0 TSval=1123301960 TSecr=3661098017|
|602469 | 0,040942 | 192.0.2.2 | 40571 | 192.0.2.1 | | 40571 → 53 [ACK] Seq=32 Ack=122 Win=64768 Len=0 TSval=3661107554 TSecr=1123301960|
|621475 | 1,959518 | 192.0.2.2 | 40571 | 192.0.2.1 | | 40571 → 53 [FIN, ACK] Seq=32 Ack=122 Win=64768 Len=0 TSval=3661109514 TSecr=1123301960|
|621476 | 0,000019 | 192.0.2.1 | 53 | 192.0.2.2 | | 53 → 40571 [ACK] Seq=122 Ack=33 Win=65280 Len=0 TSval=1123303961 TSecr=3661109514|
### Possible fixesMay 2024 (9.18.27, 9.18.27-S1, 9.19.24)https://gitlab.isc.org/isc-projects/bind9/-/issues/4469Follow-up from "Resolve "Crash on shutdown when DNSSEC validation is running:...2024-02-24T07:53:03ZMark AndrewsFollow-up from "Resolve "Crash on shutdown when DNSSEC validation is running: ENSURE(isc_mempool_getallocated(*namepoolp) == 0) failed""The following discussion from !8526 should be addressed:
- [ ] @pspacek started a [discussion](https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/8526#note_420948): (+1 comment)
> Now the hard question: Was this caused by ...The following discussion from !8526 should be addressed:
- [ ] @pspacek started a [discussion](https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/8526#note_420948): (+1 comment)
> Now the hard question: Was this caused by some recent change in the mempool usage? @ondrej?
>
> If so, can we have, say, a Cocinelle check for the correct order of operations? Chasing down these shutdown issues one by one is nightmare and consumes QA time, so if we can have an automated check I'm all for it.
>
> If an automated check is not feasible please could you manually check places affected by (presumed) recent changes to see if there are other place with similar bugs?
>
> Thank you!May 2024 (9.18.27, 9.18.27-S1, 9.19.24)https://gitlab.isc.org/isc-projects/bind9/-/issues/4502Missing reference?2024-02-24T07:53:00ZMark AndrewsMissing reference?Job [#3894124](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3894124) failed for c56a0ce25353ac1d4a8226d72373e3d7fb4c4c10:
```
2023-12-21 02:40:08 INFO:catz I:catz_tmp_8975m3uv:ns4 crashed on shutdown
2023-12-21 02:40:08 ERROR:cat...Job [#3894124](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3894124) failed for c56a0ce25353ac1d4a8226d72373e3d7fb4c4c10:
```
2023-12-21 02:40:08 INFO:catz I:catz_tmp_8975m3uv:ns4 crashed on shutdown
2023-12-21 02:40:08 ERROR:catz Failed to stop servers
2023-12-21 02:40:08 INFO:catz I:catz_tmp_8975m3uv:Core dump(s) found: /builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv/ns4/core.56121
2023-12-21 02:40:08 INFO:catz D:catz_tmp_8975m3uv:backtrace from /builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv/ns4/core.56121:
2023-12-21 02:40:08 INFO:catz D:catz_tmp_8975m3uv:--------------------------------------------------------------------------------
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:Core was generated by `/builds/isc-projects/bind9/bin/named/.libs/lt-named -D catz_tmp_8975m3uv-ns4 -m'.
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:Program terminated with signal SIGABRT, Aborted.
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#0 0x00007ff5cd7dfb8f in raise () from /lib64/libc.so.6
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:[Current thread is 1 (Thread 0x7ff5b1dff700 (LWP 56142))]
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#0 0x00007ff5cd7dfb8f in raise () from /lib64/libc.so.6
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#1 0x00007ff5cd7b2ea5 in abort () from /lib64/libc.so.6
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#2 0x0000000000422b8a in assertion_failed (file=0x7ff5d17eda82 "view.c", line=427, type=isc_assertiontype_insist, cond=0x7ff5d17c47c0 "__v > 0 && __v < (4294967295U)") at main.c:234
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#3 0x00007ff5d1c35f9d in isc_assertion_failed (file=file@entry=0x7ff5d17eda82 "view.c", line=line@entry=427, type=type@entry=isc_assertiontype_insist, cond=cond@entry=0x7ff5d17c47c0 "__v > 0 && __v < (4294967295U)") at assertions.c:48
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#4 0x00007ff5d176d241 in dns_view_attach (source=source@entry=0x7ff5ca3c0c00, targetp=targetp@entry=0x7ff5afa051f8) at view.c:429
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#5 0x000000000042b579 in catz_run (entry=0x7ff5b803f0c0, origin=origin@entry=0x7ff5ca20c540, view=0x7ff5ca3c0c00, udata=0x680608 <ns_catz_cbdata>, type=type@entry=CATZ_DELZONE) at server.c:2957
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#6 0x000000000042b5d2 in catz_delzone (entry=<optimized out>, origin=origin@entry=0x7ff5ca20c540, view=<optimized out>, udata=<optimized out>) at server.c:2973
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#7 0x00007ff5d1644645 in dns__catz_zones_merge (catz=0x7ff5ca20c540, newcatz=0x7ff5afa17000) at catz.c:696
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#8 0x00007ff5d1647be0 in dns__catz_update_cb (data=<optimized out>) at catz.c:2481
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#9 0x00007ff5d1c6316a in isc__work_cb (req=<optimized out>) at work.c:30
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#10 0x00007ff5cf6244ee in worker () from /lib64/libuv.so.1
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#11 0x00007ff5ce4f71da in start_thread () from /lib64/libpthread.so.0
2023-12-21 02:40:09 INFO:catz D:/builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv:#12 0x00007ff5cd7cae73 in clone () from /lib64/libc.so.6
2023-12-21 02:40:09 INFO:catz D:catz_tmp_8975m3uv:--------------------------------------------------------------------------------
2023-12-21 02:40:09 INFO:catz D:catz_tmp_8975m3uv:full backtrace from /builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv/ns4/core.56121 saved in /builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv/ns4/core.56121-backtrace.txt
2023-12-21 02:40:10 INFO:catz D:catz_tmp_8975m3uv:core dump /builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv/ns4/core.56121 archived as /builds/isc-projects/bind9/bin/tests/system/catz_tmp_8975m3uv/ns4/core.56121.gz
2023-12-21 02:40:11 INFO:catz I:catz_tmp_8975m3uv:1 assertion failure(s) found
2023-12-21 02:40:11 ERROR:catz Found core dumps or sanitizer reports
2023-12-21 02:40:11 INFO:catz test artifacts in: catz_sh_catz
```May 2024 (9.18.27, 9.18.27-S1, 9.19.24)https://gitlab.isc.org/isc-projects/bind9/-/issues/4564slightly worse cold cache performance after the KeyTrap fix2024-02-24T07:52:45ZTom Krizekslightly worse cold cache performance after the KeyTrap fixThe keytrap fix (merged in isc-private/bind9!628) has a significant impact on client query latency during startup (cold cache) as well as increased memory consumption. Using `recursive-clients 1000;` yielded quite similar results to `100...The keytrap fix (merged in isc-private/bind9!628) has a significant impact on client query latency during startup (cold cache) as well as increased memory consumption. Using `recursive-clients 1000;` yielded quite similar results to `10000` (which was used for most charts in here).
## Cold cache latency
### :question: 9.18 cold cache UDP
![keytrap-cold-cache-latency-9.18](/uploads/9953460d5a50fb571f18c21ece5f7415/keytrap-cold-cache-latency-9.18.png)
#### load 15x
| | before | after |
| ------ | ------ | ----- |
| responses <2.0s | 91 % | 79 % |
| responses <100ms | 88 % | 75 % |
#### load 10x
| | before | after |
| ------ | ------ | ----- |
| responses <2.0s | 96 % | 89 % |
| responses <100ms | 92 % | 85 % |
#### load 5x
| | before | after |
| ------ | ------ | ----- |
| responses <2.0s | 99 % | 98 % |
| responses <100ms | 93 % | 92 % |
### :question: 9.16 cold cache UDP
![keytrap-cold-cache-latency-9.16](/uploads/795943f2c6c6959c15f30c9f41597976/keytrap-cold-cache-latency-9.16.png)
### :white_check_mark: 9.19 cold cache UDP
the impact for ~"v9.19" is quite minimal and I don't consider it an issue
![keytrap-cold-cache-latency-9.19](/uploads/d86be8aeb8170b8e596bd85666937a81/keytrap-cold-cache-latency-9.19.png)
### :question: 9.18 cold cache TCP
The performance drop can also be observed with TCP with much lower overall throughput.
![keytrap-hot-cache-latency-tcp-9.18](/uploads/b581e94d77c5c1cdc9e143fdfff3bac2/keytrap-hot-cache-latency-tcp-9.18.png)
## :white_check_mark: Hot cache latency
The good news is that it doesn't really affect performance with hot cache.
![keytrap-hot-cache-latency-9.18](/uploads/e1704c0c853dc5b6ab798e1821b4a7db/keytrap-hot-cache-latency-9.18.png)
## Memory consumption
### :question: 9.18 memory consumption UDP
The initial memory consumption also gets slightly higher:
![keytrap-memory-9.18](/uploads/7d89b5dbc594c64c57802e11418c03b6/keytrap-memory-9.18.png)
### :white_check_mark: 9.18 memory consumption TCP
While the memory consumption is slightly higher for TCP under load, I think this can be explained by the fact that some queries take longer time to resolve -> some connections might be open for a longer time, thus consume more resources than before.
![keytrap-memory-tcp-9.18](/uploads/88be13f12e69bb69fc7f6d917dc21e6f/keytrap-memory-tcp-9.18.png)May 2024 (9.18.27, 9.18.27-S1, 9.19.24)