BIND issueshttps://gitlab.isc.org/isc-projects/bind9/-/issues2023-12-06T18:35:00Zhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4462Crash on shutdown when DNSSEC validation is running: ENSURE(isc_mempool_getal...2023-12-06T18:35:00ZPetr Špačekpspacek@isc.orgCrash on shutdown when DNSSEC validation is running: ENSURE(isc_mempool_getallocated(*namepoolp) == 0) failed### Summary
(Summarize the bug encountered concisely.)
### BIND version used
* ~"Affects v9.19": 235659b95ad53fd51fa90105b17ba1a4e51df5b0
Does not affect:
* ~"v9.18": 6817bf1284fe8aea303365d2dd17bc5523e7a41b
* ~"v9.16": 161d69aba357f...### Summary
(Summarize the bug encountered concisely.)
### BIND version used
* ~"Affects v9.19": 235659b95ad53fd51fa90105b17ba1a4e51df5b0
Does not affect:
* ~"v9.18": 6817bf1284fe8aea303365d2dd17bc5523e7a41b
* ~"v9.16": 161d69aba357fa830bb6ef2b097b0447929041f0
* ~"v9.11 (EoL)": v9.11.37-S1
* Other versions were not tested
### Steps to reproduce
Essentially cause validator to work on something during shutdown. One possibility is simply random subdomain attack against a signed zone.
1. Run an auth:
- zone: [local.testiscorg.ch.zone.signed](/uploads/e33d65b59661293a9a2722c992b9edd7/local.testiscorg.ch.zone.signed)
- config: [auth.conf](/uploads/adcd06cde487e5da3f24dd6359084e2e/auth.conf)
- `named -g -c auth.conf`
2. Run `named` under attack:
- [resolver.conf](/uploads/7c06f9959cfaf7601a89623ae2efffe4/resolver.conf)
- `named -g -c resolver.conf -n1 -D resolver`
The `-n1` makes it easier to trigger.
3. Run random subdomain attack:
- [randnames.py](/uploads/ee30deec22e8da98fa2949b91ae54ce7/randnames.py)
- `python randlabels.py | dnsperf -s 127.0.0.1 -S1 -D`
4. SIGINT the resolver:
- `pkill -f resolver`
### What is the current *bug* behavior?
:boom:
```
message.c:4768: ENSURE(isc_mempool_getallocated(*namepoolp) == 0) failed
```
<details>
```
(gdb) bt
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1 0x00007ffff6bea8a3 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
#2 0x00007ffff6b9a668 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3 0x00007ffff6b824b8 in __GI_abort () at abort.c:79
#4 0x000055555557e3b2 in assertion_failed (file=0x7ffff7eae157 "message.c", line=4768, type=isc_assertiontype_ensure, cond=0x7ffff7eaf818 "isc_mempool_getallocated(*namepoolp) == 0") at main.c:234
#5 0x00007ffff7f502ea in isc_assertion_failed (file=0x7ffff7eae157 "message.c", line=4768, type=isc_assertiontype_ensure, cond=0x7ffff7eaf818 "isc_mempool_getallocated(*namepoolp) == 0") at assertions.c:48
#6 0x00007ffff7cf9a59 in dns_message_destroypools (namepoolp=0x7ffff3e38580, rdspoolp=0x7ffff3e38588) at message.c:4768
#7 0x00007ffff7de87c3 in dns_resolver__destroy (res=0x7ffff3e31c00) at resolver.c:9892
#8 0x00007ffff7dea2b4 in dns_resolver_unref (ptr=0x7ffff3e31c00) at resolver.c:10173
#9 0x00007ffff7dea38d in dns_resolver_detach (ptrp=0x7ffff3e6e848) at resolver.c:10173
#10 0x00007ffff7c6ed52 in destroy (adb=0x7ffff3e6e800) at adb.c:1830
#11 0x00007ffff7c6ef20 in dns_adb_unref (ptr=0x7ffff3e6e800) at adb.c:1838
#12 0x00007ffff7c6eff9 in dns_adb_detach (ptrp=0x7fffffff9dc0) at adb.c:1838
#13 0x00007ffff7e26a56 in dns_view_detach (viewp=0x7ffff13b3e08) at view.c:516
#14 0x00007ffff7e237dd in destroy_validator (val=0x7ffff13b3e00) at validator.c:3122
#15 0x00007ffff7e23f49 in dns_validator_unref (ptr=0x7ffff13b3e00) at validator.c:3226
#16 0x00007ffff7e24022 in dns_validator_detach (ptrp=0x7fffffff9fa0) at validator.c:3226
#17 0x00007ffff7e1c421 in validator_done_cb (arg=0x7ffff13b3e00) at validator.c:211
#18 0x00007ffff7f507ac in isc__async_cb (handle=0x7ffff3e90388) at async.c:111
#19 0x00007ffff78dba1b in uv__async_io (loop=0x7ffff3e90020, w=<optimized out>, events=<optimized out>) at src/unix/async.c:176
#20 0x00007ffff78f8d48 in uv__io_poll (loop=0x7ffff3e90020, timeout=<optimized out>) at src/unix/linux.c:1526
#21 0x00007ffff78e0fbf in uv_run (loop=0x7ffff3e90020, mode=UV_RUN_DEFAULT) at src/unix/core.c:447
#22 0x00007ffff7f6de2c in loop_thread (arg=0x7ffff3e90000) at loop.c:282
#23 0x00007ffff7f847fd in thread_body (wrap=0x7ffff3ee59c0) at thread.c:85
#24 0x00007ffff7f848b6 in isc_thread_main (func=0x7ffff7f6dcb2 <loop_thread>, arg=0x7ffff3e90000) at thread.c:116
#25 0x00007ffff7f6eead in isc_loopmgr_run (loopmgr=0x7ffff3e206c0) at loop.c:454
#26 0x00005555555810e2 in main (argc=5, argv=0x7fffffffe598) at main.c:1574
```
</details>
### What is the expected *correct* behavior?
No crash.December 2023 (9.18.21, 9.18.21-S1, 9.19.19)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/4457dig crashes after SIGINT if there are multiple queries2023-12-04T21:59:30ZPetr Špačekpspacek@isc.orgdig crashes after SIGINT if there are multiple queries### Summary
Dig with multiple queries crashes when interrupted.
### BIND version used
* ~"Affects v9.19" : de2009e3c2a
### Steps to reproduce
```
$ dig 2000.delay.getdnsapi.net 3000.delay.getdnsapi.net
```
+ SIGINT before the first qu...### Summary
Dig with multiple queries crashes when interrupted.
### BIND version used
* ~"Affects v9.19" : de2009e3c2a
### Steps to reproduce
```
$ dig 2000.delay.getdnsapi.net 3000.delay.getdnsapi.net
```
+ SIGINT before the first query finishes.
### What is the current *bug* behavior?
:boom:
```
signal.c:78: REQUIRE(((signal) != ((void *)0) && ((const isc__magic_t *)(signal))->magic == ((('S') << 24 | ('I') << 16 | ('G') << 8 | (' '))))) failed, back trace
```
### What is the expected *correct* behavior?
No crash.
### Relevant configuration files
None needed.
### Relevant logs and/or screenshots
Full debug log:
<details>
```
$ dig -d 2000.delay.getdnsapi.net 3000.delay.getdnsapi.net
setup_libs()
setup_system()
create_search_list()
ndots is 1.
timeout is 0.
retries is 3.
get_server_list()
make_server(127.0.0.111)
dig_query_setup
parse_args()
making new lookup
make_empty_lookup()
make_empty_lookup() = 0x7f6faa9aa000->references = 1
digrc (open)
main parsing +timeout=5
main parsing +retry=0
main parsing -d
main parsing 2000.delay.getdnsapi.net
clone_lookup()
make_empty_lookup()
make_empty_lookup() = 0x7f6faa9ab800->references = 1
clone_server_list()
looking up 2000.delay.getdnsapi.net
main parsing 3000.delay.getdnsapi.net
clone_lookup()
make_empty_lookup()
make_empty_lookup() = 0x7f6faa9ad000->references = 1
clone_server_list()
looking up 3000.delay.getdnsapi.net
dig_startup()
start_lookup()
setup_lookup(0x7f6faa9ab800)
resetting lookup counter.
cloning server list
clone_server_list()
make_server(127.0.0.111)
idn_textname: 2000.delay.getdnsapi.net
using root origin
recursive query
AD query
add_question()
starting to render the message
add_opt()
done rendering
create query 0x7f6fab09f540 linked to lookup 0x7f6faa9ab800
dighost.c:2141:lookup_attach(0x7f6faa9ab800) = 2
dighost.c:2652:new_query(0x7f6fab09f540) = 1
do_lookup()
start_udp(0x7f6fab09f540)
dighost.c:3255:query_attach(0x7f6fab09f540) = 2
working on lookup 0x7f6faa9ab800, query 0x7f6fab09f540
dighost.c:3300:query_attach(0x7f6fab09f540) = 3
udp_ready()
udp_ready(0x7f6fab09f700, success, 0x7f6fab09f540)
dighost.c:3147:lookup_attach(0x7f6faa9ab800) = 3
dighost.c:3216:query_attach(0x7f6fab09f540) = 4
recving with lookup=0x7f6faa9ab800, query=0x7f6fab09f540, handle=0x7f6fab09f700
recvcount=1
have local timeout of 5000
dighost.c:3094:query_attach(0x7f6fab09f540) = 5
sending a request
sendcount=1
dighost.c:1729:query_detach(0x7f6fab09f540) = 4
dighost.c:3236:query_detach(0x7f6fab09f540) = 3
dighost.c:3237:lookup_detach(0x7f6faa9ab800) = 2
send_done(0x7f6fab09f700, success, 0x7f6fab09f540)
sendcount=0
dighost.c:2729:lookup_attach(0x7f6faa9ab800) = 3
dighost.c:2746:query_detach(0x7f6fab09f540) = 2
dighost.c:2747:lookup_detach(0x7f6faa9ab800) = 2
check_if_done()
list full
pending lookup 0x7f6faa9ad000
^Crecv_done(0x7f6fab09f700, shutting down, 0x7ffe1fb53710, 0x7f6fab09f540)
recvcount=0
dighost.c:3905:lookup_attach(0x7f6faa9ab800) = 3
recv_done: cancel
dighost.c:3913:_cancel_lookup()
canceling pending query 0x7f6fab09f540, belonging to 0x7f6faa9ab800
dighost.c:2775:query_detach(0x7f6fab09f540) = 1
check_if_done()
list full
pending lookup 0x7f6faa9ad000
dighost.c:3915:query_detach(0x7f6fab09f540) = 0
dighost.c:3915:destroy_query(0x7f6fab09f540) = 0
dighost.c:1687:lookup_detach(0x7f6faa9ab800) = 2
dighost.c:3916:lookup_detach(0x7f6faa9ab800) = 1
clear_current_lookup()
lookup cleared
dighost.c:1820:lookup_detach(0x7f6faa9ab800) = 0
destroy_lookup
freeing server 0x7f6fab072a00 belonging to 0x7f6faa9ab800
start_lookup()
setup_lookup(0x7f6faa9ad000)
resetting lookup counter.
cloning server list
clone_server_list()
make_server(127.0.0.111)
idn_textname: 3000.delay.getdnsapi.net
using root origin
recursive query
AD query
add_question()
starting to render the message
add_opt()
done rendering
create query 0x7f6fab09f540 linked to lookup 0x7f6faa9ad000
dighost.c:2141:lookup_attach(0x7f6faa9ad000) = 2
dighost.c:2652:new_query(0x7f6fab09f540) = 1
do_lookup()
start_udp(0x7f6fab09f540)
dighost.c:3255:query_attach(0x7f6fab09f540) = 2
working on lookup 0x7f6faa9ad000, query 0x7f6fab09f540
signal.c:78: REQUIRE(((signal) != ((void *)0) && ((const isc__magic_t *)(signal))->magic == ((('S') << 24 | ('I') << 16 | ('G') << 8 | (' '))))) failed, back trace
/usr/lib/libisc-9.19.19-dev.so(+0x33891)[0x7f6faef42891]
/usr/lib/libisc-9.19.19-dev.so(isc_assertion_failed+0x31)[0x7f6faef427a2]
/usr/lib/libisc-9.19.19-dev.so(isc_signal_stop+0x44)[0x7f6faef71f5b]
/usr/lib/libisc-9.19.19-dev.so(isc_loopmgr_blocking+0x55)[0x7f6faef61f96]
dig(get_address+0x38)[0x55ddd5ed030b]
dig(+0x1b006)[0x55ddd5ecc006]
dig(do_lookup+0xc8)[0x55ddd5ed067b]
dig(start_lookup+0x285)[0x55ddd5ec6e71]
dig(+0x15485)[0x55ddd5ec6485]
dig(+0x15fcc)[0x55ddd5ec6fcc]
dig(+0x1d23b)[0x55ddd5ece23b]
/usr/lib/libisc-9.19.19-dev.so(+0x1e71d)[0x7f6faef2d71d]
/usr/lib/libisc-9.19.19-dev.so(isc__nm_readcb+0x121)[0x7f6faef2d863]
/usr/lib/libisc-9.19.19-dev.so(isc__nm_udp_failed_read_cb+0x12b)[0x7f6faef420b0]
/usr/lib/libisc-9.19.19-dev.so(isc__nm_failed_read_cb+0x89)[0x7f6faef2ac2c]
/usr/lib/libisc-9.19.19-dev.so(isc__nm_udp_shutdown+0x127)[0x7f6faef42723]
/usr/lib/libisc-9.19.19-dev.so(isc__nmsocket_shutdown+0x6e)[0x7f6faef2dd24]
/usr/lib/libisc-9.19.19-dev.so(+0x1edb4)[0x7f6faef2ddb4]
/usr/lib/libuv.so.1(uv_walk+0x9b)[0x7f6fae9a474b]
/usr/lib/libisc-9.19.19-dev.so(+0x1818b)[0x7f6faef2718b]
/usr/lib/libisc-9.19.19-dev.so(isc__async_cb+0x18d)[0x7f6faef42c6b]
/usr/lib/libuv.so.1(+0x9a1b)[0x7f6fae99fa1b]
/usr/lib/libuv.so.1(+0x26d48)[0x7f6fae9bcd48]
/usr/lib/libuv.so.1(uv_run+0x1bf)[0x7f6fae9a4fbf]
/usr/lib/libisc-9.19.19-dev.so(+0x51370)[0x7f6faef60370]
/usr/lib/libisc-9.19.19-dev.so(+0x66f07)[0x7f6faef75f07]
/usr/lib/libisc-9.19.19-dev.so(isc_thread_main+0x62)[0x7f6faef75fc6]
/usr/lib/libisc-9.19.19-dev.so(isc_loopmgr_run+0x187)[0x7f6faef613fb]
dig(dig_startup+0x48)[0x55ddd5ec0624]
dig(main+0x40)[0x55ddd5ec068a]
/usr/lib/libc.so.6(+0x27cd0)[0x7f6fae9f1cd0]
/usr/lib/libc.so.6(__libc_start_main+0x8a)[0x7f6fae9f1d8a]
dig(_start+0x25)[0x55ddd5eb7045]
Aborted (core dumped)
```
</details>December 2023 (9.18.21, 9.18.21-S1, 9.19.19)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/4451Cache overmem setting is not reset2023-12-06T18:29:37ZMark AndrewsCache overmem setting is not resetSet a low cache size then send the server a query stream. Once the cache fills the server does not recover.
```
options {
listen-on port 5555 { 127.0.0.1; };
listen-on-v6 port 5555 { ::1; };
pid-file none;
...Set a low cache size then send the server a query stream. Once the cache fills the server does not recover.
```
options {
listen-on port 5555 { 127.0.0.1; };
listen-on-v6 port 5555 { ::1; };
pid-file none;
max-cache-size 1M;
};
```December 2023 (9.18.21, 9.18.21-S1, 9.19.19)https://gitlab.isc.org/isc-projects/bind9/-/issues/4418rbtdb.c:582: INSIST(!cds_lfht_destroy(rbtdb->common.update_listeners, ((void ...2023-11-14T16:06:32ZMichal Nowakrbtdb.c:582: INSIST(!cds_lfht_destroy(rbtdb->common.update_listeners, ((void *)0))) failedEmploying `rr` chaos mode on system tests of the `main` branch, I got a shutdown crash in `catz`:
```
Core was generated by `/home/newman/isc/ws/bind9/bin/named/.libs/named -D catz_tmp_8__h0mh7-ns4 -m rec'. ...Employing `rr` chaos mode on system tests of the `main` branch, I got a shutdown crash in `catz`:
```
Core was generated by `/home/newman/isc/ws/bind9/bin/named/.libs/named -D catz_tmp_8__h0mh7-ns4 -m rec'.
Program terminated with signal SIGABRT, Aborted.
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0;
[Current thread is 1 (Thread 0x756a4d1e8680 (LWP 187005))]
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1 0x00007ff3f03228f3 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78 #2 0x00007ff3f02d1afe in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3 0x00007ff3f02ba87f in __GI_abort () at abort.c:79
#4 0x0000000000417b4a in assertion_failed (file=0x7fffea179e8b "rbtdb.c", line=582, type=isc_assertiontype_insist, cond=0x7fffea178fb8 "!cds_lfht_destroy(rbtdb->common.update_listeners, ((void *)0))") at main.c:234
#5 0x000022cf17dba56a in isc_assertion_failed (file=file@entry=0x7fffea179e8b "rbtdb.c", line=line@entry=582, type=type@entry=isc_assertiontype_insist, cond=cond@entry=0x7fffea178fb8 "!cds_lfht_destroy(rbtdb->common.update_listeners, ((void *)0))") at assertions.c:48
#6 0x00007fffea086b75 in free_rbtdb (rbtdb=rbtdb@entry=0x12675bcbd000, log=log@entry=true) at rbtdb.c:582
#7 0x00007fffea0879fe in dns__rbtdb_destroy (arg=0x12675bcbd000) at rbtdb.c:646
#8 0x00007fffea00d708 in dns__catz_done_cb (data=0x3aff36a446c0) at catz.c:2527
#9 0x000022cf17de247d in isc__after_work_cb (req=<optimized out>, status=0) at work.c:42
#10 0x00007ff3f02268a9 in uv__work_done (handle=0x3aff36a77a50) at src/threadpool.c:329
#11 0x00007ff3f021de63 in uv__async_io (loop=0x3aff36a779a0, w=<optimized out>, events=<optimized out>) at src/unix/async.c:176
#12 0x00007ff3f023bfae in uv__io_poll (loop=0x3aff36a779a0, timeout=<optimized out>) at src/unix/linux.c:1476
#13 0x00007ff3f0223558 in uv_run (loop=loop@entry=0x3aff36a779a0, mode=mode@entry=UV_RUN_DEFAULT) at src/unix/core.c:447
#14 0x000022cf17dcce8c in loop_thread (arg=arg@entry=0x3aff36a77980) at loop.c:282
#15 0x000022cf17ddc1d1 in thread_body (wrap=0x3aff36a9b2e0) at thread.c:85
#16 thread_run (wrap=0x3aff36a9b2e0) at thread.c:100
#17 0x00007ff3f0320947 in start_thread (arg=<optimized out>) at pthread_create.c:444
#18 0x00007ff3f03a6764 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100
```
[core.187002-backtrace.txt](/uploads/9403d7b3b309b58248bf58361c766c80/core.187002-backtrace.txt)
[named.run](/uploads/abf3926d07d2f5571e3c1153cbcec69c/named.run)
[core.187002.gz](/uploads/ccf6c2cbe6165d1e7be26cf3f78a00c1/core.187002.gz)
[rr_trace.txz](/uploads/9fb36e52080e2d97c45c0a3c78e33d3d/rr_trace.txz) (`rr pack` for `rr replay`, if needed, from Fedora 38 but should work everywhere)December 2023 (9.18.21, 9.18.21-S1, 9.19.19)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4414shutdown crash in control_recvmessage(): INSIST(!conn->shuttingdown)2023-12-06T18:25:29ZMichal Nowakshutdown crash in control_recvmessage(): INSIST(!conn->shuttingdown)Job [#3776566](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3776566) failed for 7d650fde89ab2306eaf520c10370ae7a45b6b640.
This is a `main` shutdown crash in `control_recvmessage()` of `bin/named/controlconf.c`: `INSIST(!conn->shutti...Job [#3776566](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3776566) failed for 7d650fde89ab2306eaf520c10370ae7a45b6b640.
This is a `main` shutdown crash in `control_recvmessage()` of `bin/named/controlconf.c`: `INSIST(!conn->shuttingdown);`.
```
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:Core was generated by `/builds/isc-projects/bind9/workspace/bin/named/.libs/named -c /builds/isc-proje'.
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:Program terminated with signal SIGABRT, Aborted.
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:Download failed: Invalid argument. Continuing without source file ./nptl/./nptl/pthread_kill.c.
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:[Current thread is 1 (Thread 0x7fdcd5101500 (LWP 377769))]
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#1 0x00007fdcd7c42d9f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#2 0x00007fdcd7bf3f32 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#3 0x00007fdcd7bde472 in __GI_abort () at ./stdlib/abort.c:79
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#4 0x0000564f929377b5 in assertion_failed (file=<optimized out>, line=410, type=isc_assertiontype_insist, cond=0x564f92972bf0 "!conn->shuttingdown") at /builds/isc-projects/bind9/bin/named/main.c:234
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#5 0x00007fdcd8886649 in isc_assertion_failed (file=file@entry=0x564f929721e0 "/builds/isc-projects/bind9/bin/named/controlconf.c", line=line@entry=410, type=type@entry=isc_assertiontype_insist, cond=cond@entry=0x564f92972bf0 "!conn->shuttingdown") at /builds/isc-projects/bind9/lib/isc/assertions.c:48
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#6 0x0000564f92933b05 in control_recvmessage (handle=<optimized out>, result=result@entry=ISC_R_SHUTTINGDOWN, arg=<optimized out>) at /builds/isc-projects/bind9/bin/named/controlconf.c:410
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#7 0x0000564f92933f39 in shutdown_listener (listener=<optimized out>) at /builds/isc-projects/bind9/bin/named/controlconf.c:204
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#8 0x0000564f9293411b in controls_shutdown (controls=controls@entry=0x7fdcd4c141a0) at /builds/isc-projects/bind9/bin/named/controlconf.c:642
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#9 0x0000564f92934422 in named_controls_shutdown (controls=0x7fdcd4c141a0) at /builds/isc-projects/bind9/bin/named/controlconf.c:648
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#10 0x0000564f929599de in shutdown_server (arg=0x7fdcd4c9f700) at /builds/isc-projects/bind9/bin/named/server.c:9896
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#11 0x00007fdcd8886976 in isc__async_cb (handle=<optimized out>) at /builds/isc-projects/bind9/lib/isc/async.c:111
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#12 0x00007fdcd8089749 in uv__async_io (loop=0x7fdcd4cae820, w=0x7fdcd4cae9e8, events=1) at /usr/src/libuv-v1.46.0/src/unix/async.c:176
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#13 0x00007fdcd80a4f9b in uv__io_poll (loop=0x7fdcd4cae820, timeout=29999) at /usr/src/libuv-v1.46.0/src/unix/linux.c:1476
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#14 0x00007fdcd808a62c in uv_run (loop=0x7fdcd4cae820, mode=UV_RUN_DEFAULT) at /usr/src/libuv-v1.46.0/src/unix/core.c:447
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#15 0x00007fdcd8899b24 in loop_thread (arg=arg@entry=0x7fdcd4cae800) at /builds/isc-projects/bind9/lib/isc/loop.c:282
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#16 0x00007fdcd88a89ba in thread_body (wrap=0x564f9304a1f0) at /builds/isc-projects/bind9/lib/isc/thread.c:85
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#17 0x00007fdcd88a8a34 in isc_thread_main (func=func@entry=0x7fdcd8899a99 <loop_thread>, arg=0x7fdcd4cae800) at /builds/isc-projects/bind9/lib/isc/thread.c:116
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#18 0x00007fdcd889a817 in isc_loopmgr_run (loopmgr=0x7fdcd4c696c0) at /builds/isc-projects/bind9/lib/isc/loop.c:454
2023-11-03 00:23:07 INFO:shutdown D:/builds/isc-projects/bind9/workspace/bin/tests/system/shutdown_tmp_f2a3q9qs:#19 0x0000564f9293a0ee in main (argc=<optimized out>, argv=<optimized out>) at /builds/isc-projects/bind9/bin/named/main.c:1574
```
[core.377769-backtrace.txt](/uploads/5e442072838418b35b9601d4c92aee5a/core.377769-backtrace.txt)
[named.run](/uploads/01a56e1fddc884569ab5c210b6718d8f/named.run)December 2023 (9.18.21, 9.18.21-S1, 9.19.19)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4384Assertion failure at ENSURE(isc_mempool_getallocated(*namepoolp) == 0) in mes...2023-12-05T12:50:50ZMichal NowakAssertion failure at ENSURE(isc_mempool_getallocated(*namepoolp) == 0) in message.c:4791See #4462 for reproducer.
Job [#3740189](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3740189) failed for cddd9dcb5329165633767fd1e319fbb0700487d2; worked fine 2 MRs prior (at 2728b810). Looks like a shutdown issue.
```
Core was ge...See #4462 for reproducer.
Job [#3740189](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3740189) failed for cddd9dcb5329165633767fd1e319fbb0700487d2; worked fine 2 MRs prior (at 2728b810). Looks like a shutdown issue.
```
Core was generated by `/builds/isc-projects/bind9/.local/usr/local/sbin/named -f -c ./named.conf'.
Program terminated with signal SIGABRT, Aborted.
#0 __pthread_kill_implementation (threadid=281472872136768, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
Downloading source file /usr/src/debug/glibc-2.37-10.fc38.aarch64/nptl/pthread_kill.c...
44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0;
[Current thread is 1 (Thread 0xffff828ec040 (LWP 24241))]
#0 __pthread_kill_implementation (threadid=281472872136768, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1 0x0000ffff8171d958 [PAC] in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
#2 0x0000ffff816d4980 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3 0x0000ffff816c0284 [PAC] in __GI_abort () at abort.c:79
#4 0x000000000042efa8 [PAC] in assertion_failed (file=<optimized out>, line=4791, type=isc_assertiontype_ensure, cond=0xffff827a2368 "isc_mempool_getallocated(*namepoolp) == 0") at main.c:234
#5 0x0000ffff828658a4 in isc_assertion_failed (file=file@entry=0xffff827a0998 "message.c", line=line@entry=4791, type=type@entry=isc_assertiontype_ensure, cond=cond@entry=0xffff827a2368 "isc_mempool_getallocated(*namepoolp) == 0") at assertions.c:48
#6 0x0000ffff82678060 in dns_message_destroypools (namepoolp=0xffff80bf8e20, rdspoolp=0xffff80bf8e40) at message.c:4791
#7 0x0000ffff827031ec in dns_resolver__destroy (res=res@entry=0xffff80b21400) at resolver.c:9891
#8 0x0000ffff8270b0e4 in dns_resolver_unref (ptr=0xffff80b21400) at resolver.c:10172
#9 0x0000ffff8270b178 in dns_resolver_detach (ptrp=ptrp@entry=0xffff80070050) at resolver.c:10172
#10 0x0000ffff826185dc in destroy (adb=adb@entry=0xffff80070000) at adb.c:1833
#11 0x0000ffff8261acd4 in dns_adb_unref (ptr=0xffff80070000) at adb.c:1841
#12 0x0000ffff8261ae6c in dns_adb_detach (ptrp=ptrp@entry=0xffffec576580) at adb.c:1841
#13 0x0000ffff8273bf74 in dns_view_detach (viewp=viewp@entry=0xffff8090ae08) at view.c:516
#14 0x0000ffff82738050 in destroy_validator (val=val@entry=0xffff8090ae00) at validator.c:3122
#15 0x0000ffff82738148 in dns_validator_unref (ptr=0xffff8090ae00) at validator.c:3226
#16 0x0000ffff82736020 in dns_validator_detach (ptrp=ptrp@entry=0xffffec576688) at validator.c:3226
#17 0x0000ffff82737ee4 in validator_done_cb (arg=<optimized out>) at validator.c:211
#18 0x0000ffff82865c1c in isc__async_cb (handle=<optimized out>) at async.c:111
#19 0x0000ffff81dda0c0 in uv__async_io (loop=0xffff80482820, w=0xffff804829f0, events=1) at /usr/src/libuv-v1.46.0/src/unix/async.c:176
#20 0x0000ffff81df6d0c in uv__io_poll (loop=0xffff80482820, timeout=11996) at /usr/src/libuv-v1.46.0/src/unix/linux.c:1476
#21 0x0000ffff81ddb084 in uv_run (loop=0xffff80482820, mode=UV_RUN_DEFAULT) at /usr/src/libuv-v1.46.0/src/unix/core.c:447
#22 0x0000ffff8287977c in loop_thread (arg=arg@entry=0xffff80482800) at loop.c:282
#23 0x0000ffff82889204 in thread_body (wrap=0x29878850) at thread.c:85
#24 0x0000ffff8288928c in isc_thread_main (func=func@entry=0xffff82879710 <loop_thread>, arg=<optimized out>) at thread.c:116
#25 0x0000ffff8287a4e0 in isc_loopmgr_run (loopmgr=0xffff808c06c0) at loop.c:454
#26 0x00000000004318a8 in main (argc=<optimized out>, argv=<optimized out>) at main.c:1580
```
[core.24241-backtrace.txt](/uploads/8c4ac88405f3d8fb408f3ce70c590971/core.24241-backtrace.txt)
[named.conf](/uploads/28375b11fdea2ade28463c2c3c437c01/named.conf)
Similar issue: isc-projects/bind9#2188December 2023 (9.18.21, 9.18.21-S1, 9.19.19)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/4328ThreadSanitizer: data race in dns_tsigkeyring_dump2023-12-06T18:18:29ZOndřej SurýThreadSanitizer: data race in dns_tsigkeyring_dump```
==================
WARNING: ThreadSanitizer: data race (pid=1731304)
Write of size 8 at 0x7b1800000b90 by thread T16:
#0 isc_hashmap_iter_create /home/ondrej/Projects/bind9/lib/isc/hashmap.c:608:20 (libisc-9.19.18-dev.so+0x55c1...```
==================
WARNING: ThreadSanitizer: data race (pid=1731304)
Write of size 8 at 0x7b1800000b90 by thread T16:
#0 isc_hashmap_iter_create /home/ondrej/Projects/bind9/lib/isc/hashmap.c:608:20 (libisc-9.19.18-dev.so+0x55c12) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#1 dns_tsigkeyring_dump /home/ondrej/Projects/bind9/lib/dns/tsig.c:473:2 (libdns-9.19.18-dev.so+0x2ed639) (BuildId: 0c875e4d1ebe40ad865f17a9c21753da586ae188)
#2 destroy /home/ondrej/Projects/bind9/lib/dns/view.c:239:13 (libdns-9.19.18-dev.so+0x3140c6) (BuildId: 0c875e4d1ebe40ad865f17a9c21753da586ae188)
#3 dns_view_weakdetach /home/ondrej/Projects/bind9/lib/dns/view.c:586:3 (libdns-9.19.18-dev.so+0x313b0c) (BuildId: 0c875e4d1ebe40ad865f17a9c21753da586ae188)
#4 zone_shutdown /home/ondrej/Projects/bind9/lib/dns/zone.c:14515:3 (libdns-9.19.18-dev.so+0x3642dd) (BuildId: 0c875e4d1ebe40ad865f17a9c21753da586ae188)
#5 isc__async_cb /home/ondrej/Projects/bind9/lib/isc/async.c:111:3 (libisc-9.19.18-dev.so+0x46682) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#6 uv__async_io /home/ondrej/Projects/tsan/libuv/src/unix/async.c:176:5 (libuv.so.1+0x19fb5) (BuildId: 473d7a8bf34342bf61b8e7193e565ce2b7962210)
#7 uv__io_poll /home/ondrej/Projects/tsan/libuv/src/unix/linux.c:1476:11 (libuv.so.1+0x4d289) (BuildId: 473d7a8bf34342bf61b8e7193e565ce2b7962210)
#8 uv_run /home/ondrej/Projects/tsan/libuv/src/unix/core.c:447:5 (libuv.so.1+0x1adee) (BuildId: 473d7a8bf34342bf61b8e7193e565ce2b7962210)
#9 loop_thread /home/ondrej/Projects/bind9/lib/isc/loop.c:282:6 (libisc-9.19.18-dev.so+0x79c40) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#10 thread_body /home/ondrej/Projects/bind9/lib/isc/thread.c:85:8 (libisc-9.19.18-dev.so+0x9e423) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#11 thread_run /home/ondrej/Projects/bind9/lib/isc/thread.c:100:14 (libisc-9.19.18-dev.so+0x9e70f) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
Previous write of size 8 at 0x7b1800000b90 by thread T29:
#0 isc_hashmap_iter_create /home/ondrej/Projects/bind9/lib/isc/hashmap.c:608:20 (libisc-9.19.18-dev.so+0x55c12) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#1 dns_tsigkeyring_dump /home/ondrej/Projects/bind9/lib/dns/tsig.c:473:2 (libdns-9.19.18-dev.so+0x2ed639) (BuildId: 0c875e4d1ebe40ad865f17a9c21753da586ae188)
#2 destroy /home/ondrej/Projects/bind9/lib/dns/view.c:239:13 (libdns-9.19.18-dev.so+0x3140c6) (BuildId: 0c875e4d1ebe40ad865f17a9c21753da586ae188)
#3 dns_view_weakdetach /home/ondrej/Projects/bind9/lib/dns/view.c:586:3 (libdns-9.19.18-dev.so+0x313b0c) (BuildId: 0c875e4d1ebe40ad865f17a9c21753da586ae188)
#4 zone_shutdown /home/ondrej/Projects/bind9/lib/dns/zone.c:14515:3 (libdns-9.19.18-dev.so+0x3642dd) (BuildId: 0c875e4d1ebe40ad865f17a9c21753da586ae188)
#5 isc__async_cb /home/ondrej/Projects/bind9/lib/isc/async.c:111:3 (libisc-9.19.18-dev.so+0x46682) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#6 uv__async_io /home/ondrej/Projects/tsan/libuv/src/unix/async.c:176:5 (libuv.so.1+0x19fb5) (BuildId: 473d7a8bf34342bf61b8e7193e565ce2b7962210)
#7 uv__io_poll /home/ondrej/Projects/tsan/libuv/src/unix/linux.c:1476:11 (libuv.so.1+0x4d289) (BuildId: 473d7a8bf34342bf61b8e7193e565ce2b7962210)
#8 uv_run /home/ondrej/Projects/tsan/libuv/src/unix/core.c:447:5 (libuv.so.1+0x1adee) (BuildId: 473d7a8bf34342bf61b8e7193e565ce2b7962210)
#9 loop_thread /home/ondrej/Projects/bind9/lib/isc/loop.c:282:6 (libisc-9.19.18-dev.so+0x79c40) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#10 thread_body /home/ondrej/Projects/bind9/lib/isc/thread.c:85:8 (libisc-9.19.18-dev.so+0x9e423) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#11 thread_run /home/ondrej/Projects/bind9/lib/isc/thread.c:100:14 (libisc-9.19.18-dev.so+0x9e70f) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
Location is heap block of size 88 at 0x7b1800000b40 allocated by main thread:
#0 malloc <null> (lt-named+0x738dc) (BuildId: a34b7ed3d2c9f1e42cab4821d53fbde3e4c5b946)
#1 mallocx /home/ondrej/Projects/bind9/lib/isc/./jemalloc_shim.h:67:14 (libisc-9.19.18-dev.so+0x8591e) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#2 mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:305:8 (libisc-9.19.18-dev.so+0x7f23e) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#3 isc__mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:744:8 (libisc-9.19.18-dev.so+0x7f0c6) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#4 isc_hashmap_create /home/ondrej/Projects/bind9/lib/isc/hashmap.c:210:27 (libisc-9.19.18-dev.so+0x53a20) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#5 dns_tsigkeyring_create /home/ondrej/Projects/bind9/lib/dns/tsig.c:1589:2 (libdns-9.19.18-dev.so+0x2f49c7) (BuildId: 0c875e4d1ebe40ad865f17a9c21753da586ae188)
#6 dns_view_create /home/ondrej/Projects/bind9/lib/dns/view.c:144:2 (libdns-9.19.18-dev.so+0x3121e6) (BuildId: 0c875e4d1ebe40ad865f17a9c21753da586ae188)
#7 create_view /home/ondrej/Projects/bind9/bin/named/server.c:6425:11 (lt-named+0x141562) (BuildId: a34b7ed3d2c9f1e42cab4821d53fbde3e4c5b946)
#8 load_configuration /home/ondrej/Projects/bind9/bin/named/server.c:9111:12 (lt-named+0x13a831) (BuildId: a34b7ed3d2c9f1e42cab4821d53fbde3e4c5b946)
#9 run_server /home/ondrej/Projects/bind9/bin/named/server.c:9952:2 (lt-named+0x11f465) (BuildId: a34b7ed3d2c9f1e42cab4821d53fbde3e4c5b946)
#10 isc__async_cb /home/ondrej/Projects/bind9/lib/isc/async.c:111:3 (libisc-9.19.18-dev.so+0x46682) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#11 uv__async_io /home/ondrej/Projects/tsan/libuv/src/unix/async.c:176:5 (libuv.so.1+0x19fb5) (BuildId: 473d7a8bf34342bf61b8e7193e565ce2b7962210)
#12 uv__io_poll /home/ondrej/Projects/tsan/libuv/src/unix/linux.c:1476:11 (libuv.so.1+0x4d289) (BuildId: 473d7a8bf34342bf61b8e7193e565ce2b7962210)
#13 uv_run /home/ondrej/Projects/tsan/libuv/src/unix/core.c:447:5 (libuv.so.1+0x1adee) (BuildId: 473d7a8bf34342bf61b8e7193e565ce2b7962210)
#14 loop_thread /home/ondrej/Projects/bind9/lib/isc/loop.c:282:6 (libisc-9.19.18-dev.so+0x79c40) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#15 thread_body /home/ondrej/Projects/bind9/lib/isc/thread.c:85:8 (libisc-9.19.18-dev.so+0x9e423) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#16 isc_thread_main /home/ondrej/Projects/bind9/lib/isc/thread.c:116:2 (libisc-9.19.18-dev.so+0x9e313) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#17 isc_loopmgr_run /home/ondrej/Projects/bind9/lib/isc/loop.c:454:2 (libisc-9.19.18-dev.so+0x79963) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#18 main /home/ondrej/Projects/bind9/bin/named/main.c:1580:2 (lt-named+0x115cb2) (BuildId: a34b7ed3d2c9f1e42cab4821d53fbde3e4c5b946)
Thread T16 'isc-loop-0016' (tid=1733466, running) created by main thread at:
#0 pthread_create <null> (lt-named+0x754bb) (BuildId: a34b7ed3d2c9f1e42cab4821d53fbde3e4c5b946)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/thread.c:139:8 (libisc-9.19.18-dev.so+0x9e667) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#2 isc_loopmgr_run /home/ondrej/Projects/bind9/lib/isc/loop.c:448:3 (libisc-9.19.18-dev.so+0x798e4) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#3 main /home/ondrej/Projects/bind9/bin/named/main.c:1580:2 (lt-named+0x115cb2) (BuildId: a34b7ed3d2c9f1e42cab4821d53fbde3e4c5b946)
Thread T29 'isc-loop-0029' (tid=1733524, running) created by main thread at:
#0 pthread_create <null> (lt-named+0x754bb) (BuildId: a34b7ed3d2c9f1e42cab4821d53fbde3e4c5b946)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/thread.c:139:8 (libisc-9.19.18-dev.so+0x9e667) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#2 isc_loopmgr_run /home/ondrej/Projects/bind9/lib/isc/loop.c:448:3 (libisc-9.19.18-dev.so+0x798e4) (BuildId: 39be444545292c4d1a56c1084e4150ff076e95d7)
#3 main /home/ondrej/Projects/bind9/bin/named/main.c:1580:2 (lt-named+0x115cb2) (BuildId: a34b7ed3d2c9f1e42cab4821d53fbde3e4c5b946)
SUMMARY: ThreadSanitizer: data race /home/ondrej/Projects/bind9/lib/isc/hashmap.c:608:20 in isc_hashmap_iter_create
==================
ThreadSanitizer: reported 1 warnings
```December 2023 (9.18.21, 9.18.21-S1, 9.19.19)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4465Release Checklist for BIND 9.18.21, 9.18.21-S1, 9.19.192024-01-10T07:23:06ZTom KrizekRelease Checklist for BIND 9.18.21, 9.18.21-S1, 9.19.19## Release Schedule
**Code Freeze:** Wednesday, 6 December 2023
**Tagging Deadline:** Monday, 11 December 2023
**Public Release:** Wednesday, 20 December 2023
## Documentation Review Links
**Closed issues assigned to the milestone ...## Release Schedule
**Code Freeze:** Wednesday, 6 December 2023
**Tagging Deadline:** Monday, 11 December 2023
**Public Release:** Wednesday, 20 December 2023
## Documentation Review Links
**Closed issues assigned to the milestone without a release note:**
- [9.18.21](https://gitlab.isc.org/isc-projects/bind9/-/issues?scope=all&sort=created_asc&state=closed&milestone_title=December+2023+%289.16.46%2C+9.16.46-S1%2C+9.18.21%2C+9.18.21-S1%2C+9.19.19%29¬%5Blabel_name%5D%5B%5D=Release+Notes¬%5Blabel_name%5D%5B%5D=Duplicate&label_name%5B%5D=v9.18)
- [9.18.21-S1](https://gitlab.isc.org/isc-private/bind9/-/issues?scope=all&sort=created_asc&state=closed&milestone_title=December+2023+%289.16.46%2C+9.16.46-S1%2C+9.18.21%2C+9.18.21-S1%2C+9.19.19%29¬%5Blabel_name%5D%5B%5D=Release+Notes¬%5Blabel_name%5D%5B%5D=Duplicate&label_name%5B%5D=v9.18-S)
- [9.19.19](https://gitlab.isc.org/isc-projects/bind9/-/issues?scope=all&sort=created_asc&state=closed&milestone_title=December+2023+%289.16.46%2C+9.16.46-S1%2C+9.18.21%2C+9.18.21-S1%2C+9.19.19%29¬%5Blabel_name%5D%5B%5D=Release+Notes¬%5Blabel_name%5D%5B%5D=Duplicate&label_name%5B%5D=v9.19)
**Merge requests merged into the milestone without a release note:**
- [9.18.21](https://gitlab.isc.org/isc-projects/bind9/-/merge_requests?scope=all&sort=merged_at&state=merged&milestone_title=December+2023+%289.16.46%2C+9.16.46-S1%2C+9.18.21%2C+9.18.21-S1%2C+9.19.19%29¬%5Blabel_name%5D%5B%5D=Release+Notes&target_branch=bind-9.18)
- [9.18.21-S1](https://gitlab.isc.org/isc-private/bind9/-/merge_requests?scope=all&sort=merged_at&state=merged&milestone_title=December+2023+%289.16.46%2C+9.16.46-S1%2C+9.18.21%2C+9.18.21-S1%2C+9.19.19%29¬%5Blabel_name%5D%5B%5D=Release+Notes&target_branch=bind-9.18-sub)
- [9.19.19](https://gitlab.isc.org/isc-projects/bind9/-/merge_requests?scope=all&sort=merged_at&state=merged&milestone_title=December+2023+%289.16.46%2C+9.16.46-S1%2C+9.18.21%2C+9.18.21-S1%2C+9.19.19%29¬%5Blabel_name%5D%5B%5D=Release+Notes&target_branch=main)
**Merge requests merged into the milestone without a `CHANGES` entry:**
- [9.18.21](https://gitlab.isc.org/isc-projects/bind9/-/merge_requests?scope=all&sort=merged_at&state=merged&milestone_title=December+2023+%289.16.46%2C+9.16.46-S1%2C+9.18.21%2C+9.18.21-S1%2C+9.19.19%29&label_name%5B%5D=No+CHANGES&target_branch=bind-9.18)
- [9.18.21-S1](https://gitlab.isc.org/isc-private/bind9/-/merge_requests?scope=all&sort=merged_at&state=merged&milestone_title=December+2023+%289.16.46%2C+9.16.46-S1%2C+9.18.21%2C+9.18.21-S1%2C+9.19.19%29&label_name%5B%5D=No+CHANGES&target_branch=bind-9.18-sub)
- [9.19.19](https://gitlab.isc.org/isc-projects/bind9/-/merge_requests?scope=all&sort=merged_at&state=merged&milestone_title=December+2023+%289.16.46%2C+9.16.46-S1%2C+9.18.21%2C+9.18.21-S1%2C+9.19.19%29&label_name%5B%5D=No+CHANGES&target_branch=main)
## Release Checklist
### Before the Code Freeze
- [x] ***(QA)*** Rebase -S editions on top of current open-source versions: `git checkout bind-9.18-sub && git rebase origin/bind-9.18`
- [x] ***(QA)*** [Inform](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/inform_supp_marketing.py) Support and Marketing of [impending release](https://mattermost.isc.org/isc/pl/6oeity9pxf8fmbgx43t5ofsm6a) (and give estimated release dates).
- [x] ***(QA)*** Ensure there are no permanent test failures on any platform. Check [public](https://gitlab.isc.org/isc-projects/bind9/-/pipelines?scope=all&source=schedule) and [private](https://gitlab.isc.org/isc-private/bind9/-/pipelines?scope=all&source=schedule) scheduled pipelines.
- [x] ***(QA)*** Check charts from `shotgun:*` jobs in the scheduled pipelines to verify there is no unexplained performance drop for any protocol.
- [x] ***(QA)*** Check [Perflab](https://perflab.isc.org/) to ensure there has been no unexplained drop in performance for the versions being released.
- [x] ***(QA)*** Check whether all issues assigned to the release milestone are resolved[^1].
- [x] ***(QA)*** Ensure that there are no outstanding [merge requests in the private repository](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/)[^1] (Subscription Edition only).
- [x] ***(QA)*** [Ensure](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/check_backports.py) all merge requests marked for backporting have been indeed backported.
- [x] ***(QA)*** [Announce](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/inform_code_freeze.py) ([on Mattermost](https://mattermost.isc.org/isc/pl/gdh1dzxaqbyq8dyic4gqbr5wcy)) that the code freeze is in effect.
### Before the Tagging Deadline
- [x] ***(QA)*** Inspect the current output of the `cross-version-config-tests` job to verify that no unexpected backward-incompatible change was introduced in the current release cycle.
- [x] ***(QA)*** Ensure release notes are correct, [ask Support and Marketing](https://mattermost.isc.org/isc/pl/epghkxueoiysibjjwgcya4fgey) to check them as well. [Example](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/510)
- [x] ***(QA)*** Add a release marker to `CHANGES`. Examples: [9.18](https://gitlab.isc.org/isc-projects/bind9/-/commit/f14d8ad78c0506fd4247187f2177f8eceeb6b3b9), [9.16](https://gitlab.isc.org/isc-projects/bind9/-/commit/1bcdf21874f99a00da389d723e0ad07dfd70f9f1)
- [x] ***(QA)*** Add a release marker to `CHANGES.SE` (Subscription Edition only). [Example](https://gitlab.isc.org/isc-private/bind9/-/commit/0f03d5737bcbdaa1bf713c6db1887b14938c3421)
- [x] ***(QA)*** Update BIND 9 version in `configure.ac` ([9.18+](https://gitlab.isc.org/isc-projects/bind9/-/commit/3c85ab7f4c35e6d8acef1393606002a0a8730100)) or `version` ([9.16](https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/7692/diffs?commit_id=1bcdf21874f99a00da389d723e0ad07dfd70f9f1)).
- [x] ~~***(QA)*** Rebuild `configure` using Autoconf on `docs.isc.org` (9.16).~~
- [x] ***(QA)*** Update GitLab settings for all maintained branches to disallow merging to them: [public](https://gitlab.isc.org/isc-projects/bind9/-/settings/repository), [private](https://gitlab.isc.org/isc-private/bind9/-/settings/repository)
- [x] ***(QA)*** Tag the releases in the private repository (`git tag -s -m "BIND 9.x.y" v9.x.y`).
### Before the ASN Deadline (for ASN Releases) or the Public Release Date (for Regular Releases)
- [x] ***(QA)*** Check that the formatting is correct for the HTML version of release notes.
- [x] ***(QA)*** Check that the formatting of the generated man pages is correct.
- [x] ***(QA)*** Verify GitLab CI results [for the tags](https://gitlab.isc.org/isc-private/bind9/-/pipelines?scope=tags) created and sign off on the releases to be published.
- [x] ***(QA)*** Update GitLab settings for all maintained branches to allow merging to them again: [public](https://gitlab.isc.org/isc-projects/bind9/-/settings/repository), [private](https://gitlab.isc.org/isc-private/bind9/-/settings/repository)
- [x] ***(QA)*** Prepare (using [`version_bump.py`](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/version_bump.py)) and merge MRs resetting the release notes and updating the version string for each maintained branch.
- [x] ***(QA)*** Rebase the Subscription Edition branches (including recent release prep commits) on top of the open source branches with updated version strings.
- [x] ***(QA)*** Announce (on Mattermost) that the code freeze is over.
- [x] ***(QA)*** Request signatures for the tarballs, providing their location and checksums. Ask [signers on Mattermost](https://mattermost.isc.org/isc/channels/bind-9-qa).
- [x] ***(Signers)*** Ensure that the contents of tarballs and tags are identical.
- [x] ***(Signers)*** Validate tarball checksums, sign tarballs, and upload signatures.
- [x] ***(QA)*** Verify tarball signatures and check tarball checksums again: Run `publish_bind.sh` on repo.isc.org to pre-publish.
- [x] ~~***(QA)*** Prepare the `patches/` subdirectory for each security release (if applicable).~~
- [x] ***(QA)*** Pre-publish ASN and/or Subscription Edition tarballs so that packages can be built.
- [x] ***(QA)*** [Build and test](https://gitlab.isc.org/isc-private/rpms/bind/-/pipelines/156507) ASN and/or Subscription Edition packages (in [cloudsmith branch in private repo](https://gitlab.isc.org/isc-private/rpms/bind/-/tree/cloudsmith)). [Example](https://gitlab.isc.org/isc-private/rpms/bind/-/commit/e2512f4cfaf991827a635e374e7e93b27a5f38ba)
- [x] ***(QA)*** Use the [Printing Press project](https://gitlab.isc.org/isc-private/printing-press/-/wikis/home#adding-new-documents) to prepare a [release announcement email](https://gitlab.isc.org/isc-private/printing-press/-/merge_requests/83).
- [x] ~~***(Marketing)*** Update ASN documents in the SF portal.~~
- [x] ~~***(Marketing)*** Send out ASN emails (if applicable).~~
### On the Day of Public Release
- [x] ~~***(QA)*** Wait for clearance from Security Officer to proceed with the public release (if applicable).~~
- [x] ***(QA)*** Place tarballs in public location on FTP site.
- [x] ***(QA)*** Inform Marketing of the release, providing FTP links for the published tarballs.
- [x] ***(Marketing)*** Publish links to downloads on ISC website. [Example](https://gitlab.isc.org/website/theme-staging-site/-/commit/1ac7b30b73cb03228df4cd5651fa4e774ac35625)
- [x] ***(Marketing)*** Update the BIND -S information document in SF with download links to the new versions. (If this is a security release, this will have already been done as part of the ASN process.)
- [x] ***(Marketing)*** Update the Current Software Versions document in the SF portal if any stable versions were released.
- [x] ***(Marketing)*** Send the release announcement email to the *bind-announce* mailing list (and to *bind-users* if a major release - [example](https://lists.isc.org/pipermail/bind-users/2022-January/105624.html)).
- [x] ***(Marketing)*** Announce release on social media sites.
- [x] ***(Marketing)*** Update [Wikipedia entry for BIND](https://en.wikipedia.org/wiki/BIND).
- [x] ***(Support)*** Add the new releases to the [vulnerability matrix in the Knowledge Base](https://kb.isc.org/docs/aa-00913).
- [x] ***(Support)*** Update tickets in case of waiting support customers.
- [x] ***(QA)*** Build and test any outstanding private packages in [private repo](https://gitlab.isc.org/isc-private/rpms/bind/-/tree/cloudsmith). [Example](https://gitlab.isc.org/isc-private/rpms/bind/-/commit/2007d566db81dd9dfd79e571e2f600a3bc284da4)
- [x] ***(QA)*** Build [public RPMs](https://gitlab.isc.org/isc-packages/rpms/bind). [Example commit](https://gitlab.isc.org/isc-packages/rpms/bind/-/commit/3b5e851ea7c4e3570371a4878b5461f02a44f8cc) which triggers [Copr builds](https://copr.fedorainfracloud.org/coprs/isc/) automatically
- [x] ***(SwEng)*** Build Debian/Ubuntu packages.
- [x] ***(SwEng)*** Update Docker files [here](https://gitlab.isc.org/isc-projects/bind9-docker/-/branches) and make sure push is synchronized to [GitHub](https://github.com/isc-projects/bind9-docker). [Docker Hub](https://hub.docker.com/r/internetsystemsconsortium/bind9) should pick it up automatically. [Example](https://gitlab.isc.org/isc-projects/bind9-docker/-/commit/cada7e10e9af951595c98bfffc4bd42512faac05)
- [x] ***(QA)*** Ensure all new tags are annotated and signed. `git show --show-signature v9.19.12`
- [x] ***(QA)*** Push tags for the published releases to the public repository.
- [x] ***(QA)*** Using [`merge_tag.py`](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/merge_tag.py), merge published release tags back into the their relevant development/maintenance branches.
- [x] ***(QA)*** Ensure `allow_failure: true` is removed from the `cross-version-config-tests` job if it was set during the current release cycle.
- [x] ***(QA)*** Sanitize confidential issues which are assigned to the current release milestone and do not describe a security vulnerability, then make them public.
- [x] ***(QA)*** Sanitize [confidential issues](https://gitlab.isc.org/isc-projects/bind9/-/issues/?sort=milestone_due_desc&state=opened&confidential=yes) which are assigned to older release milestones and describe security vulnerabilities, then make them public if appropriate[^2].
- [x] ***(QA)*** Update QA tools used in GitLab CI (e.g. Black, PyLint, Sphinx) by modifying the relevant [`Dockerfile`](https://gitlab.isc.org/isc-projects/images/-/merge_requests/228/diffs).
- [x] ***(QA)*** Run a pipeline to rebuild all [images](https://gitlab.isc.org/isc-projects/images) used in GitLab CI.
- [x] ***(QA)*** Update [`metadata.json`](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/metadata.json) with the upcoming release information.
[^1]: If not, use the time remaining until the tagging deadline to ensure all outstanding issues are either resolved or moved to a different milestone.
[^2]: As a rule of thumb, security vulnerabilities which have reproducers merged to the public repository are considered okay for full disclosure.December 2023 (9.18.21, 9.18.21-S1, 9.19.19)Tom KrizekTom Krizek2023-12-20https://gitlab.isc.org/isc-projects/bind9/-/issues/4407TSAN error in isc_hashmap_iter_create2023-11-07T11:00:59ZMark AndrewsTSAN error in isc_hashmap_iter_createJob [#3773372](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3773372) failed for 0482451f84b0f266dab1e2d2f2ba78ca20ec5b73:
Missing lock / atomics on iterator count.
hashmap->iterators++;
```
WARNING: ThreadSanitizer: data race
...Job [#3773372](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3773372) failed for 0482451f84b0f266dab1e2d2f2ba78ca20ec5b73:
Missing lock / atomics on iterator count.
hashmap->iterators++;
```
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T0001:
#0 isc_hashmap_iter_create lib/isc/hashmap.c:608 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#1 dns_tsigkeyring_dump lib/dns/tsig.c:472 (BuildId: cf5fbb2a379d17cf79b5c2d5fd3b945525f46b2c)
#2 destroy lib/dns/view.c:231 (BuildId: cf5fbb2a379d17cf79b5c2d5fd3b945525f46b2c)
#3 dns_view_weakdetach lib/dns/view.c:578 (BuildId: cf5fbb2a379d17cf79b5c2d5fd3b945525f46b2c)
#4 zone_shutdown lib/dns/zone.c:14527 (BuildId: cf5fbb2a379d17cf79b5c2d5fd3b945525f46b2c)
#5 isc__async_cb lib/isc/async.c:111 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#6 uv__async_io /usr/src/libuv-v1.46.0/src/unix/async.c:176 (BuildId: 797e130002e5c7a3ee5c0e181992b461482df75b)
#7 thread_body lib/isc/thread.c:85 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#8 thread_run lib/isc/thread.c:100 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
Previous write of size 8 at 0x000000000001 by thread T0002:
#0 isc_hashmap_iter_create lib/isc/hashmap.c:608 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#1 dns_tsigkeyring_dump lib/dns/tsig.c:472 (BuildId: cf5fbb2a379d17cf79b5c2d5fd3b945525f46b2c)
#2 destroy lib/dns/view.c:231 (BuildId: cf5fbb2a379d17cf79b5c2d5fd3b945525f46b2c)
#3 dns_view_weakdetach lib/dns/view.c:578 (BuildId: cf5fbb2a379d17cf79b5c2d5fd3b945525f46b2c)
#4 zone_shutdown lib/dns/zone.c:14527 (BuildId: cf5fbb2a379d17cf79b5c2d5fd3b945525f46b2c)
#5 isc__async_cb lib/isc/async.c:111 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#6 uv__async_io /usr/src/libuv-v1.46.0/src/unix/async.c:176 (BuildId: 797e130002e5c7a3ee5c0e181992b461482df75b)
#7 thread_body lib/isc/thread.c:85 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#8 thread_run lib/isc/thread.c:100 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
Location is heap block of size 88 at 0x000000000012 allocated by main thread:
#0 malloc <null> (BuildId: 78ee840c3fb04c49132c060982867c27b52d0ffb)
#1 mallocx lib/isc/jemalloc_shim.h:67 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#2 mem_get lib/isc/mem.c:305
#3 isc__mem_get lib/isc/mem.c:744 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#4 isc_hashmap_create lib/isc/hashmap.c:210 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#5 dns_tsigkeyring_create lib/dns/tsig.c:1588 (BuildId: cf5fbb2a379d17cf79b5c2d5fd3b945525f46b2c)
#6 dns_view_create lib/dns/view.c:144 (BuildId: cf5fbb2a379d17cf79b5c2d5fd3b945525f46b2c)
#7 create_view bin/named/server.c:6423 (BuildId: 7bf7fd2425148f507bca7f65a93860eed28a6abc)
#8 load_configuration bin/named/server.c:9018 (BuildId: 7bf7fd2425148f507bca7f65a93860eed28a6abc)
#9 run_server bin/named/server.c:9859 (BuildId: 7bf7fd2425148f507bca7f65a93860eed28a6abc)
#10 isc__async_cb lib/isc/async.c:111 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#11 uv__async_io /usr/src/libuv-v1.46.0/src/unix/async.c:176 (BuildId: 797e130002e5c7a3ee5c0e181992b461482df75b)
#12 thread_body lib/isc/thread.c:85 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#13 isc_thread_main lib/isc/thread.c:116 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#14 isc_loopmgr_run lib/isc/loop.c:454 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#15 main bin/named/main.c:1574 (BuildId: 7bf7fd2425148f507bca7f65a93860eed28a6abc)
Thread T0001 'isc-loop-0002' (running) created by main thread at:
#0 pthread_create <null> (BuildId: 78ee840c3fb04c49132c060982867c27b52d0ffb)
#1 isc_thread_create lib/isc/thread.c:139 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#2 isc_loopmgr_run lib/isc/loop.c:448 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#3 main bin/named/main.c:1574 (BuildId: 7bf7fd2425148f507bca7f65a93860eed28a6abc)
Thread T0002 'isc-loop-0003' (running) created by main thread at:
#0 pthread_create <null> (BuildId: 78ee840c3fb04c49132c060982867c27b52d0ffb)
#1 isc_thread_create lib/isc/thread.c:139 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#2 isc_loopmgr_run lib/isc/loop.c:448 (BuildId: 7ec65504b258e2240d31f1743404da3227b2aed3)
#3 main bin/named/main.c:1574 (BuildId: 7bf7fd2425148f507bca7f65a93860eed28a6abc)
SUMMARY: ThreadSanitizer: data race lib/isc/hashmap.c:608 in isc_hashmap_iter_create
```December 2023 (9.18.21, 9.18.21-S1, 9.19.19)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4513System tests fail with Net::DNS 1.422024-01-03T01:01:34ZMark AndrewsSystem tests fail with Net::DNS 1.42Net::DNS::Nameserver->main_loop no longer loops. This breaks reclimit and chain system tests which use Net::DNS::Nameserver in ans.pl.Net::DNS::Nameserver->main_loop no longer loops. This breaks reclimit and chain system tests which use Net::DNS::Nameserver in ans.pl.January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/4500Log the change that generated "not exact" when applying a diff.2024-01-04T16:56:21ZMark AndrewsLog the change that generated "not exact" when applying a diff.Provide more information a "not exact" response is detected. Log name, class, type and operation being attempted.Provide more information a "not exact" response is detected. Log name, class, type and operation being attempted.January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)https://gitlab.isc.org/isc-projects/bind9/-/issues/4498[GL #4494] followup: regression test was too strict2024-01-04T16:58:18ZMark Andrews[GL #4494] followup: regression test was too strictThe delta which records the addition of the private record for the NSEC3 to NSEC conversion can sometimes not be the first delta. Update the system test to handle it in a later delta.
https://gitlab.isc.org/isc-projects/bind9/-/jobs/38...The delta which records the addition of the private record for the NSEC3 to NSEC conversion can sometimes not be the first delta. Update the system test to handle it in a later delta.
https://gitlab.isc.org/isc-projects/bind9/-/jobs/3883570January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)https://gitlab.isc.org/isc-projects/bind9/-/issues/4497Kindly mute the 'trust-anchor-telemetry' experimental warning.2023-12-18T14:15:22ZJakub MocKindly mute the 'trust-anchor-telemetry' experimental warning.<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confident...<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confidential by clicking the checkbox at the bottom!
-->
### Summary
<!-- Concisely summarize the bug encountered. -->
Kindly mute the 'trust-anchor-telemetry' experimental warning.
### BIND version affected
<!--
Make sure you are testing with the **latest** supported version of BIND
for a given branch. Many bugs have been fixed over time!
See https://kb.isc.org/docs/supported-platforms for the current list.
The latest source is available from https://www.isc.org/download/#BIND
Paste the output of `named -V` here.
-->
```
BIND 9.18.20 (Extended Support Version) <id:>
running on FreeBSD amd64 13.2-RELEASE-p7 FreeBSD 13.2-RELEASE-p7 stable/23.7-n254871-d5ec322cffc SMP
built by make with '--disable-linux-caps' '--localstatedir=/var' '--sysconfdir=/usr/local/etc/namedb' '--with-dlopen=yes' '--with-libxml2' '--with-openssl=/usr/local' '--enable-dnsrps' '--with-readline=libedit' '--enable-dnstap' '--disable-fixed-rrset' '--disable-geoip' '--without-maxminddb' '--without-gssapi' '--with-libidn2=/usr/local' '--with-json-c' '--disable-largefile' '--with-lmdb=/usr/local' '--disable-querytrace' '--enable-tcp-fastopen' '--prefix=/usr/local' '--mandir=/usr/local/man' '--disable-silent-rules' '--infodir=/usr/local/share/info/' '--build=amd64-portbld-freebsd13.2' 'build_alias=amd64-portbld-freebsd13.2' 'CC=cc' 'CFLAGS=-O2 -pipe -DLIBICONV_PLUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing ' 'LDFLAGS= -L/usr/local/lib -ljson-c -Wl,-rpath,/usr/local/lib -fstack-protector-strong ' 'LIBS=-L/usr/local/lib' 'CPPFLAGS=-DLIBICONV_PLUG -isystem /usr/local/include' 'CPP=cpp' 'PKG_CONFIG=pkgconf' 'PKG_CONFIG_LIBDIR=/usr/obj/usr/ports/dns/bind918/work/.pkgconfig:/usr/local/libdata/pkgconfig:/usr/local/share/pkgconfig:/usr/libdata/pkgconfig' 'READLINE_CFLAGS=-L/usr/local/lib'
compiled by CLANG FreeBSD Clang 14.0.5 (https://github.com/llvm/llvm-project.git llvmorg-14.0.5-0-gc12386ae247c)
compiled with OpenSSL version: OpenSSL 1.1.1w 11 Sep 2023
linked to OpenSSL version: OpenSSL 1.1.1w 11 Sep 2023
compiled with libuv version: 1.47.0
linked to libuv version: 1.47.0
compiled with libnghttp2 version: 1.58.0
linked to libnghttp2 version: 1.58.0
compiled with libxml2 version: 2.10.4
linked to libxml2 version: 21004
compiled with json-c version: 0.17
linked to json-c version: 0.17
compiled with zlib version: 1.2.13
linked to zlib version: 1.2.13
compiled with protobuf-c version: 1.4.1
linked to protobuf-c version: 1.4.1
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): no
default paths:
named configuration: /usr/local/etc/namedb/named.conf
rndc configuration: /usr/local/etc/namedb/rndc.conf
DNSSEC root key: /usr/local/etc/namedb/bind.keys
nsupdate session key: /var/run/named/session.key
named PID file: /var/run/named/pid
named lock file: /var/run/named/named.lock
```
### Steps to reproduce
<!--
This is extremely important! Be precise and use itemized lists, please.
Even if a default configuration is affected, please include the full configuration
files _you were testing with_.
Example:
1. Use _attached_ configuration file
2. Start BIND server with command: `named -g -c named.conf ...`
3. Simulate legitimate clients using command `dnsperf -S1 -d legit-queries ...`
4. Simulate attack traffic using command `dnsperf -S1 -d attack-queries ...`
-->
1. Use _attached_ configuration file and start BIND server
### What is the current *bug* behavior?
<!-- What actually happens. -->
So this "experimental" features has been introduced about 5 years ago. Yet, after all that time, one or two warning lines are logged to syslog with `LOG_WARNING` severity, depending on whether you foolishly tried to mute the annoying hardcoded warning with the `trust-anchor-telemetry no;` option [as suggested in KB](https://kb.isc.org/docs/aa-01528). The hardcoded warning is annoying, doubling the pointless noise when you try to disable the feature - and, if fact, with a configuration that completely ignores DNSSEC since BIND is only used here to filter out AAAA for certain domains to avoid geolocation with IPv6 tunnels with certain domains - is just inexplicable.
### What is the expected *correct* behavior?
<!-- What you should see instead. -->
Do not log pointless warnings to syslog.
### Relevant configuration files
<!-- Paste any relevant configuration files here - please use code blocks (```)
to format console output. If submitting the contents of your
configuration file in a non-confidential issue, it is advisable to
obscure key secrets; this can be done automatically by using
`named-checkconf -px`. -->
```
acl "Allow_ACL" {
127.0.0.0/8;
};
controls {
inet 127.0.0.1 port 9530 allow {
127.0.0.1/32;
} keys {
"rndc-key";
};
};
logging {
channel "default_log" {
file "/var/log/named/named.log" versions 3 size 5242880;
print-time yes;
print-severity yes;
print-category yes;
};
channel "query_log" {
file "/var/log/named/query.log" versions 3 size 5242880;
print-time yes;
};
channel "rpz_log" {
file "/var/log/named/rpz.log" versions 3 size 5242880;
print-time yes;
};
category "default" {
"default_log";
};
category "general" {
"default_log";
};
category "queries" {
"query_log";
};
category "rpz" {
"rpz_log";
};
category "lame-servers" {
"null";
};
};
options {
directory "/usr/local/etc/namedb/working";
dump-file "/var/dump/named_dump.db";
listen-on port 53530 {
127.0.0.1/32;
};
listen-on-v6 port 53530 {
::1/128;
};
pid-file "/var/run/named/pid";
statistics-file "/var/stats/named.stats";
allow-recursion {
"Allow_ACL";
};
dnssec-validation no;
max-cache-size 80%;
recursion yes;
allow-query {
"Allow_ACL";
};
};
key "rndc-key" {
algorithm "hmac-sha256";
secret "????????????????????????????????????????????";
};
plugin query "/usr/local/lib/bind/filter-aaaa.so" {
filter-aaaa-on-v4 break-dnssec;
filter-aaaa-on-v6 break-dnssec;
};
zone "." {
type hint;
file "/usr/local/etc/namedb/named.root";
};
zone "localhost" {
type primary;
file "/usr/local/etc/namedb/primary/localhost-forward.db";
};
zone "127.in-addr.arpa" {
type primary;
file "/usr/local/etc/namedb/primary/localhost-reverse.db";
};
zone "0.ip6.arpa" {
type primary;
file "/usr/local/etc/namedb/primary/localhost-reverse.db";
};
```
### Relevant logs
<!-- Paste any relevant logs here - please use code blocks (```) to format console
output, logs, and code, as it's very hard to read otherwise. -->
```
<28>1 2023-12-18T08:21:44+01:00 gw.example.com named 57351 - [meta sequenceId="31"] /usr/local/etc/namedb/named.conf:27: option 'trust-anchor-telemetry' is experimental and subject to change in the future
<28>1 2023-12-18T08:21:44+01:00 gw.example.com named 57351 - [meta sequenceId="30"] config.c: option 'trust-anchor-telemetry' is experimental and subject to change in the future
```January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)https://gitlab.isc.org/isc-projects/bind9/-/issues/4495Conversion from NSEC3 to NSEC removes the NSEC3PARAM too early2024-02-23T11:40:30ZMark AndrewsConversion from NSEC3 to NSEC removes the NSEC3PARAM too earlyThe NSEC3PARAM was being removed immediately by `dns_nsec3param_deletechains` rather than waiting for the NSEC chain to be generated then removing it as part of the clean up. This could result in named returning unsigned answers which w...The NSEC3PARAM was being removed immediately by `dns_nsec3param_deletechains` rather than waiting for the NSEC chain to be generated then removing it as part of the clean up. This could result in named returning unsigned answers which would not validate as secure. This state was transitory being corrected when the NSEC chain completed building.January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)https://gitlab.isc.org/isc-projects/bind9/-/issues/4494add_sigs was using the wrong time in kasp mode2023-12-20T01:13:44ZMark Andrewsadd_sigs was using the wrong time in kasp mode`add_sigs` in lib/dns/zone.c and lib/dns/update.c with `kasp` was using `inception` as a proxy for `now`.
This resulted in RRSIGs not being generated for new keys. It could also result in the wrong keys being used.
I was fixing the `ns...`add_sigs` in lib/dns/zone.c and lib/dns/update.c with `kasp` was using `inception` as a proxy for `now`.
This resulted in RRSIGs not being generated for new keys. It could also result in the wrong keys being used.
I was fixing the `nsec3-to-nsec` test in `autosign` to actually convert from NSEC3 to NSEC and noted that the
change was not signed when it should have been as the zone was signed in the setup phase.
```
14-Dec-2023 18:07:01.331 del nsec3-to-nsec.example. 300 IN SOA mname1. . 2009102722 20 20 1814400 3600
14-Dec-2023 18:07:01.331 del nsec3-to-nsec.example. 0 IN NSEC3PARAM 1 0 0 BEEF
14-Dec-2023 18:07:01.331 add nsec3-to-nsec.example. 300 IN SOA mname1. . 2009102723 20 20 1814400 3600
14-Dec-2023 18:07:01.331 add nsec3-to-nsec.example. 0 IN TYPE65534 \# 8 000140000002BEEF
```
There are other issues that need to be address with this but lets clear this one first.January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/4488Memory/reference leak in lib/dns/zone.c:zone_sign2024-01-04T17:01:47ZMark AndrewsMemory/reference leak in lib/dns/zone.c:zone_signWhen fixing #4466 named was reporting a memory leak on shutdown. This was traced to a misplaced `continue` in `sign_zone` resulting in `dst_key's` not being freed.When fixing #4466 named was reporting a memory leak on shutdown. This was traced to a misplaced `continue` in `sign_zone` resulting in `dst_key's` not being freed.January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)https://gitlab.isc.org/isc-projects/bind9/-/issues/4478Redefinition of 'hmac' as different kind of symbol on NetBSD2024-01-03T14:36:13ZMichal NowakRedefinition of 'hmac' as different kind of symbol on NetBSDSince renaming `hmacname` to `hmac` in ffacf0aec6ac265bba307f4ef5f4915406607b7a BIND 9.19 won't build on NetBSD as this platform already defines `hmac` in the `/usr/include/stdlib.h` header:
```
CC dig.o
In file included from dig....Since renaming `hmacname` to `hmac` in ffacf0aec6ac265bba307f4ef5f4915406607b7a BIND 9.19 won't build on NetBSD as this platform already defines `hmac` in the `/usr/include/stdlib.h` header:
```
CC dig.o
In file included from dig.c:45:
./dighost.h:262:24: error: redefinition of 'hmac' as different kind of symbol
extern dst_algorithm_t hmac;
^
/usr/include/stdlib.h:312:10: note: previous definition is here
ssize_t hmac(const char *, const void *, size_t, const void *, size_t, void *,
^
dig.c:2461:9: error: non-object type 'ssize_t (const char *, const void *, size_t, const void *, size_t, void *, size_t)' (aka 'long (const char *, const void *, unsigned long, const void *, unsigned long, void *, unsigned long)') is not assignable
hmac = DST_ALG_HMACMD5;
~~~~ ^
2 errors generated.
```
This erorr is on NetBSD 10.0 RC1 but `hmac(3)` suggests this will fail on every NetBSD since v8:
```
NAME
hmac – compute a key-Hash Message Authentication Code
...
HISTORY
The hmac() function appeared in NetBSD 8.
```January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/4477statschannel test intermittently fails with incorrect zone loadtime2023-12-18T09:40:39ZTom Krizekstatschannel test intermittently fails with incorrect zone loadtimeAfter #3983 was fixed and the `loadtime` check was re-enabled, the following tests occasionally [fail](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3848463) because the `loadtime` retrieved over stats channel is 0.
- `statschannel/t...After #3983 was fixed and the `loadtime` check was re-enabled, the following tests occasionally [fail](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3848463) because the `loadtime` retrieved over stats channel is 0.
- `statschannel/tests_xml.py::test_zone_timers_secondary_xml`
- `statschannel/tests_json.py::test_zone_timers_secondary_json`
```
________________________ test_zone_timers_secondary_xml ________________________
[gw1] linux -- Python 3.11.2 /usr/bin/python3
/builds/isc-projects/bind9/bin/tests/system/statschannel/tests_xml.py:118: in test_zone_timers_secondary_xml
generic.test_zone_timers_secondary(
/builds/isc-projects/bind9/bin/tests/system/statschannel/generic.py:101: in test_zone_timers_secondary
check_zone_timers(loaded, expires, refresh, mtime)
/builds/isc-projects/bind9/bin/tests/system/statschannel/generic.py:56: in check_zone_timers
check_loaded(loaded, loaded_exp, now)
/builds/isc-projects/bind9/bin/tests/system/statschannel/generic.py:45: in check_loaded
assert loaded == expected
E assert datetime.datetime(2023, 12, 6, 0, 28, 43) == datetime.datetime(1970, 1, 1, 0, 0)
```January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4467Numerical statistics are truncated to 32-bits on export2024-01-04T16:47:06ZPetr Špačekpspacek@isc.orgNumerical statistics are truncated to 32-bits on export### Summary
In BIND statistics, values larger than 4294967295 overflow during export. E.g. any server which processes more 4294967295 queries will see nonsense in statistics. This can conceivably happen in practice within a single day i...### Summary
In BIND statistics, values larger than 4294967295 overflow during export. E.g. any server which processes more 4294967295 queries will see nonsense in statistics. This can conceivably happen in practice within a single day if server is handling sustained ~ 50 k QPS.
Internally tracking still works up to 2^63-1, i.e. 9223372036854775807, but there is no way to get the data out without using debugger.
### BIND version used
Broken by 4e5edb35e475e4868ccbb8e4796b3fbe8ac90bb7, MR !1493.
* ~"Affects v9.19": f8fece81bf651275c3914d2559717943228a4cfd
* ~"Affects v9.18": acf55e125e946f39df96aca26608b01c46968a7b
* ~"Affects v9.16": 161d69aba357fa830bb6ef2b097b0447929041f0
### Steps to reproduce
It's kinda lengthy. Just do 2^32 queries and check /json/v1/server opcodes[] stats to see if they ever exceed 2^32-1.
### What is the current *bug* behavior?
Counters are not monotonic because of the overflow during export.
### What is the expected *correct* behavior?
No information loss.January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)Petr Špačekpspacek@isc.orgPetr Špačekpspacek@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/4466CDS is stuck on an old key.2024-01-12T08:57:15ZBjörn PerssonCDS is stuck on an old key.### Summary
The zone rombobeorn.se seems to be stuck in a CSK rollover that never gets finished. The CDS record still specifies the old key. Thus the parent zone doesn't update DS. Thus the old DNSKEY record can't be removed.
### BIND ...### Summary
The zone rombobeorn.se seems to be stuck in a CSK rollover that never gets finished. The CDS record still specifies the old key. Thus the parent zone doesn't update DS. Thus the old DNSKEY record can't be removed.
### BIND version used
```
# named -V
BIND 9.18.19-1~deb12u1-Debian (Extended Support Version) <id:>
running on Linux x86_64 5.10.0-26-amd64 #1 SMP Debian 5.10.197-1 (2023-09-29)
built by make with '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/x86_64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--disable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=yes' '--with-libidn2' '--with-json-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' '--enable-dnstap' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -ffile-prefix-map=/build/reproducible-path/bind9-9.18.19=. -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
compiled by GCC 12.2.0
compiled with OpenSSL version: OpenSSL 3.0.10 1 Aug 2023
linked to OpenSSL version: OpenSSL 3.0.11 19 Sep 2023
compiled with libuv version: 1.44.2
linked to libuv version: 1.44.2
compiled with libnghttp2 version: 1.52.0
linked to libnghttp2 version: 1.52.0
compiled with libxml2 version: 2.9.14
linked to libxml2 version: 20914
compiled with json-c version: 0.16
linked to json-c version: 0.16
compiled with zlib version: 1.2.13
linked to zlib version: 1.2.13
linked to maxminddb version: 1.7.1
compiled with protobuf-c version: 1.4.1
linked to protobuf-c version: 1.4.1
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): yes
default paths:
named configuration: /etc/bind/named.conf
rndc configuration: /etc/bind/rndc.conf
DNSSEC root key: /etc/bind/bind.keys
nsupdate session key: //run/named/session.key
named PID file: //run/named/named.pid
named lock file: //run/named/named.lock
geoip-directory: /usr/share/GeoIP
```
### Steps to reproduce
This zone has successfully replaced one CSK with another before. This was the DNSsec policy at the time:
```
dnssec-policy "automatik" {
keys {
csk lifetime P1M algorithm rsasha256 2048;
};
};
```
In an attempt to decrease the time the zone spends with dual keys, I changed the policy to this:
```
dnssec-policy "automatik" {
keys {
csk lifetime P1M algorithm rsasha256 2048;
};
dnskey-ttl P1D;
max-zone-ttl P1D;
signatures-validity P1W;
signatures-refresh P2D;
};
```
### What is the current *bug* behavior?
On 2023-11-20 it was time for another rollover. CSK 58364 was generated and published in a second DNSKEY record. The DNSKEY, CDS and CDNSKEY records were signed with both the old and the new key. Other records had their signatures replaced gradually. Since 2023-12-01 all the records except DNSKEY, CDS and CDNSKEY have signatures only by the new key. Yet CDS and CDNSKEY still show the old key, 44674. You can check it yourself:
```
$ dig +short CDS rombobeorn.se
44674 8 2 DC0A35038C492439E044C0A109A62A7447427B606104613D7BA4B32D 2EDAC3FB
```
On 2023-12-02 it was time to renew the signatures for DNSKEY, CDS and CDNSKEY. They were again signed with both keys. There are still dual DNSKEY records.
Validation still succeeds, presumably because the new key is signed with the old key. Bind seems to understand that it can't remove the old key yet, but it's not publishing a CDS record for the new key.
### What is the expected *correct* behavior?
If the policy I have configured is wrong somehow, then it should have been rejected with an informative error message. Otherwise the CDS record (and CDNSKEY) should have been changed to 58364 by now.
### Relevant configuration files
```
options {
directory "/var/cache/bind";
dnssec-validation auto;
key-directory "/var/lib/bind";
listen-on-v6 { any; };
};
dnssec-policy "som_det_var" {
keys {
ksk lifetime unlimited algorithm rsasha256 2048;
zsk lifetime unlimited algorithm rsasha256 2048;
};
dnskey-ttl P1D;
purge-keys 0;
};
dnssec-policy "automatik" {
keys {
csk lifetime P1M algorithm rsasha256 2048;
};
dnskey-ttl P1D;
max-zone-ttl P1D;
signatures-validity P1W;
signatures-refresh P2D;
};
view "internal" {
match-clients { [omitted] };
recursion yes;
allow-recursion { [omitted] };
allow-transfer { [omitted] };
notify no;
include "/etc/bind/named.conf.default-zones";
include "/etc/bind/zones.rfc1918";
zone "xn--rombobjrn-67a.se" {
type master;
file "/var/lib/bind/db.xn--rombobjrn-67a.se.internal";
dnssec-policy automatik;
parental-agents { ::1; };
inline-signing no;
update-policy { [omitted] };
};
zone "rombobeorn.se" {
type master;
file "/var/lib/bind/db.rombobeorn.se.internal";
dnssec-policy automatik;
parental-agents { ::1; };
inline-signing no;
update-policy { [omitted] };
};
zone "168.192.in-addr.arpa" {
type master;
file "/var/lib/bind/db.168.192";
update-policy { [omitted] };
};
};
view "external" {
match-clients {
any;
};
recursion no;
allow-transfer { [omitted] };
also-notify { [omitted] };
notify explicit;
rate-limit {
responses-per-second 4;
slip 2;
};
zone "xn--rombobjrn-67a.se" {
type master;
file "/var/lib/bind/db.xn--rombobjrn-67a.se.external";
dnssec-policy automatik;
parental-agents { ::1; };
inline-signing no;
update-policy { [omitted] };
};
zone "rombobeorn.se" {
type master;
file "/var/lib/bind/db.rombobeorn.se.external";
dnssec-policy automatik;
parental-agents { ::1; };
inline-signing no;
update-policy { [omitted] };
};
};
```
### Relevant logs and/or screenshots
As a baseline, these messages about two previously retired keys were repeated every hour:
```
2023-11-20T04:05:53.076358+01:00 cutie named[443161]: zone rombobeorn.se/IN/internal: reconfiguring zone keys
2023-11-20T04:05:53.105296+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-20T04:05:53.105790+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-20T04:05:53.109375+01:00 cutie named[443161]: zone rombobeorn.se/IN/internal: next key event: 20-Nov-2023 05:04:58.070
2023-11-20T04:05:53.206927+01:00 cutie named[443161]: zone rombobeorn.se/IN/external: reconfiguring zone keys
2023-11-20T04:05:53.237195+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-20T04:05:53.237622+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-20T04:05:53.241190+01:00 cutie named[443161]: zone rombobeorn.se/IN/external: next key event: 20-Nov-2023 05:04:58.202
```
Then the new key was generated, and a message about that key was added to the hourly repeats:
```
2023-11-20T05:04:58.076407+01:00 cutie named[443161]: zone rombobeorn.se/IN/internal: reconfiguring zone keys
2023-11-20T05:04:58.105335+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-20T05:04:58.105847+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-20T05:05:02.049479+01:00 cutie named[443161]: keymgr: DNSKEY rombobeorn.se/RSASHA256/58364 (CSK) created for policy automatik
2023-11-20T05:05:02.057591+01:00 cutie named[443161]: Fetching rombobeorn.se/RSASHA256/58364 (CSK) from key repository.
2023-11-20T05:05:02.058067+01:00 cutie named[443161]: DNSKEY rombobeorn.se/RSASHA256/58364 (CSK) is now published
2023-11-20T05:05:02.136470+01:00 cutie named[443161]: zone rombobeorn.se/IN/internal: next key event: 20-Nov-2023 06:04:58.070
2023-11-20T05:05:02.137062+01:00 cutie named[443161]: zone rombobeorn.se/IN/external: reconfiguring zone keys
2023-11-20T05:05:02.160374+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-20T05:05:02.160830+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-20T05:05:02.163720+01:00 cutie named[443161]: Fetching rombobeorn.se/RSASHA256/58364 (CSK) from key repository.
2023-11-20T05:05:02.164031+01:00 cutie named[443161]: DNSKEY rombobeorn.se/RSASHA256/58364 (CSK) is now published
2023-11-20T05:05:02.242215+01:00 cutie named[443161]: zone rombobeorn.se/IN/external: next key event: 20-Nov-2023 06:05:02.134
2023-11-20T06:04:58.076558+01:00 cutie named[443161]: zone rombobeorn.se/IN/internal: reconfiguring zone keys
2023-11-20T06:04:58.118725+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-20T06:04:58.119254+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-20T06:04:58.123463+01:00 cutie named[443161]: DNSKEY rombobeorn.se/RSASHA256/58364 (CSK) is now inactive
2023-11-20T06:04:58.125009+01:00 cutie named[443161]: zone rombobeorn.se/IN/internal: next key event: 20-Nov-2023 07:04:58.072
2023-11-20T06:05:02.140288+01:00 cutie named[443161]: zone rombobeorn.se/IN/external: reconfiguring zone keys
2023-11-20T06:05:02.183496+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-20T06:05:02.183962+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-20T06:05:02.188257+01:00 cutie named[443161]: DNSKEY rombobeorn.se/RSASHA256/58364 (CSK) is now inactive
2023-11-20T06:05:02.189804+01:00 cutie named[443161]: zone rombobeorn.se/IN/external: next key event: 20-Nov-2023 07:05:02.136
```
25 hours later the messages started appearing every ten minutes, claiming falsely that CDS and CDNSKEY had been updated:
```
2023-11-21T06:09:58.116346+01:00 cutie named[443161]: zone rombobeorn.se/IN/internal: reconfiguring zone keys
2023-11-21T06:09:58.158703+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-21T06:09:58.159244+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-21T06:09:58.171210+01:00 cutie named[443161]: CDS for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-21T06:09:58.171736+01:00 cutie named[443161]: CDNSKEY for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-21T06:09:58.178814+01:00 cutie named[443161]: zone rombobeorn.se/IN/external: reconfiguring zone keys
2023-11-21T06:09:58.219756+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-21T06:09:58.222083+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-21T06:09:58.223643+01:00 cutie named[443161]: CDS for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-21T06:09:58.223970+01:00 cutie named[443161]: CDNSKEY for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-21T06:19:58.172402+01:00 cutie named[443161]: zone rombobeorn.se/IN/internal: reconfiguring zone keys
2023-11-21T06:19:58.214907+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-21T06:19:58.215409+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-21T06:19:58.220135+01:00 cutie named[443161]: CDS for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-21T06:19:58.220718+01:00 cutie named[443161]: CDNSKEY for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-21T06:19:58.222976+01:00 cutie named[443161]: zone rombobeorn.se/IN/external: reconfiguring zone keys
2023-11-21T06:19:58.261817+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-21T06:19:58.262297+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-21T06:19:58.265388+01:00 cutie named[443161]: CDS for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-21T06:19:58.265739+01:00 cutie named[443161]: CDNSKEY for key rombobeorn.se/RSASHA256/58364 is now published
```
After another six days a message about the old key was added:
```
2023-11-27T07:24:01.188353+01:00 cutie named[443161]: zone rombobeorn.se/IN/internal: reconfiguring zone keys
2023-11-27T07:24:01.231048+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-27T07:24:01.231673+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-27T07:24:01.249383+01:00 cutie named[443161]: DNSKEY rombobeorn.se/RSASHA256/44674 (CSK) is now inactive
2023-11-27T07:24:01.250501+01:00 cutie named[443161]: CDS for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-27T07:24:01.250919+01:00 cutie named[443161]: CDNSKEY for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-27T07:24:01.253391+01:00 cutie named[443161]: zone rombobeorn.se/IN/external: reconfiguring zone keys
2023-11-27T07:24:01.287956+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-27T07:24:01.288376+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-27T07:24:01.290787+01:00 cutie named[443161]: DNSKEY rombobeorn.se/RSASHA256/44674 (CSK) is now inactive
2023-11-27T07:24:01.291398+01:00 cutie named[443161]: CDS for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-27T07:24:01.291732+01:00 cutie named[443161]: CDNSKEY for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-27T07:34:01.256339+01:00 cutie named[443161]: zone rombobeorn.se/IN/internal: reconfiguring zone keys
2023-11-27T07:34:01.299042+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-27T07:34:01.299657+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-27T07:34:01.303256+01:00 cutie named[443161]: DNSKEY rombobeorn.se/RSASHA256/44674 (CSK) is now inactive
2023-11-27T07:34:01.304268+01:00 cutie named[443161]: CDS for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-27T07:34:01.304659+01:00 cutie named[443161]: CDNSKEY for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-27T07:34:01.307121+01:00 cutie named[443161]: zone rombobeorn.se/IN/external: reconfiguring zone keys
2023-11-27T07:34:01.346113+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/50640 (ZSK)
2023-11-27T07:34:01.346504+01:00 cutie named[443161]: keymgr: retire DNSKEY rombobeorn.se/RSASHA256/48019 (KSK)
2023-11-27T07:34:01.348830+01:00 cutie named[443161]: DNSKEY rombobeorn.se/RSASHA256/44674 (CSK) is now inactive
2023-11-27T07:34:01.349518+01:00 cutie named[443161]: CDS for key rombobeorn.se/RSASHA256/58364 is now published
2023-11-27T07:34:01.349707+01:00 cutie named[443161]: CDNSKEY for key rombobeorn.se/RSASHA256/58364 is now published
```
Those messages are still being repeated every ten minutes, and that "inactive" key is still current according to CDS.
Here's what the key states were right after the new key was generated at 2023-11-20 05:05:
```
; This is the state of key 44674, for rombobeorn.se.
Algorithm: 8
Length: 2048
Lifetime: 2678400
Predecessor: 26869
Successor: 58364
KSK: yes
ZSK: yes
Generated: 20231020040458 (Fri Oct 20 06:04:58 2023)
Published: 20231020040458 (Fri Oct 20 06:04:58 2023)
Active: 20231020060958 (Fri Oct 20 08:09:58 2023)
Retired: 20231120060958 (Mon Nov 20 07:09:58 2023)
Removed: 20231126071458 (Sun Nov 26 08:14:58 2023)
DSPublish: 20231020160958 (Fri Oct 20 18:09:58 2023)
PublishCDS: 20231020060958 (Fri Oct 20 08:09:58 2023)
DSPubCount: 1
DNSKEYChange: 20231020060958 (Fri Oct 20 08:09:58 2023)
ZRRSIGChange: 20231030071458 (Mon Oct 30 08:14:58 2023)
KRRSIGChange: 20231020060958 (Fri Oct 20 08:09:58 2023)
DSChange: 20231021180958 (Sat Oct 21 20:09:58 2023)
DNSKEYState: omnipresent
ZRRSIGState: omnipresent
KRRSIGState: omnipresent
DSState: omnipresent
GoalState: hidden
```
```
; This is the state of key 58364, for rombobeorn.se.
Algorithm: 8
Length: 2048
Lifetime: 2678400
Predecessor: 44674
KSK: yes
ZSK: yes
Generated: 20231120040458 (Mon Nov 20 05:04:58 2023)
Published: 20231120040458 (Mon Nov 20 05:04:58 2023)
Active: 20231120060958 (Mon Nov 20 07:09:58 2023)
Retired: 20231221060958 (Thu Dec 21 07:09:58 2023)
Removed: 20231227071458 (Wed Dec 27 08:14:58 2023)
PublishCDS: 20231121050958 (Tue Nov 21 06:09:58 2023)
DNSKEYChange: 20231120040458 (Mon Nov 20 05:04:58 2023)
ZRRSIGChange: 20231120040458 (Mon Nov 20 05:04:58 2023)
KRRSIGChange: 20231120040458 (Mon Nov 20 05:04:58 2023)
DSChange: 20231120040458 (Mon Nov 20 05:04:58 2023)
DNSKEYState: rumoured
ZRRSIGState: hidden
KRRSIGState: rumoured
DSState: hidden
GoalState: omnipresent
```
At 2023-11-21 06:09 the new key's state looked like this:
```
; This is the state of key 58364, for rombobeorn.se.
Algorithm: 8
Length: 2048
Lifetime: 2678400
Predecessor: 44674
KSK: yes
ZSK: yes
Generated: 20231120040458 (Mon Nov 20 05:04:58 2023)
Published: 20231120040458 (Mon Nov 20 05:04:58 2023)
Active: 20231120060958 (Mon Nov 20 07:09:58 2023)
Retired: 20231221060958 (Thu Dec 21 07:09:58 2023)
Removed: 20231227071458 (Wed Dec 27 08:14:58 2023)
PublishCDS: 20231121050958 (Tue Nov 21 06:09:58 2023)
DNSKEYChange: 20231121050958 (Tue Nov 21 06:09:58 2023)
ZRRSIGChange: 20231121050958 (Tue Nov 21 06:09:58 2023)
KRRSIGChange: 20231121050958 (Tue Nov 21 06:09:58 2023)
DSChange: 20231121050958 (Tue Nov 21 06:09:58 2023)
DNSKEYState: omnipresent
ZRRSIGState: rumoured
KRRSIGState: omnipresent
DSState: rumoured
GoalState: omnipresent
```
Then, at 2023-11-27 07:24 the key states changed into this:
```
; This is the state of key 44674, for rombobeorn.se.
Algorithm: 8
Length: 2048
Lifetime: 2678400
Predecessor: 26869
Successor: 58364
KSK: yes
ZSK: yes
Generated: 20231020040458 (Fri Oct 20 06:04:58 2023)
Published: 20231020040458 (Fri Oct 20 06:04:58 2023)
Active: 20231020060958 (Fri Oct 20 08:09:58 2023)
Retired: 20231120060958 (Mon Nov 20 07:09:58 2023)
Removed: 20231126071458 (Sun Nov 26 08:14:58 2023)
DSPublish: 20231020160958 (Fri Oct 20 18:09:58 2023)
PublishCDS: 20231020060958 (Fri Oct 20 08:09:58 2023)
DSPubCount: 1
DNSKEYChange: 20231020060958 (Fri Oct 20 08:09:58 2023)
ZRRSIGChange: 20231127062401 (Mon Nov 27 07:24:01 2023)
KRRSIGChange: 20231020060958 (Fri Oct 20 08:09:58 2023)
DSChange: 20231021180958 (Sat Oct 21 20:09:58 2023)
DNSKEYState: omnipresent
ZRRSIGState: unretentive
KRRSIGState: omnipresent
DSState: omnipresent
GoalState: hidden
```
```
; This is the state of key 58364, for rombobeorn.se.
Algorithm: 8
Length: 2048
Lifetime: 2678400
Predecessor: 44674
KSK: yes
ZSK: yes
Generated: 20231120040458 (Mon Nov 20 05:04:58 2023)
Published: 20231120040458 (Mon Nov 20 05:04:58 2023)
Active: 20231120060958 (Mon Nov 20 07:09:58 2023)
Retired: 20231221060958 (Thu Dec 21 07:09:58 2023)
Removed: 20231227071458 (Wed Dec 27 08:14:58 2023)
PublishCDS: 20231121050958 (Tue Nov 21 06:09:58 2023)
DNSKEYChange: 20231121050958 (Tue Nov 21 06:09:58 2023)
ZRRSIGChange: 20231127062401 (Mon Nov 27 07:24:01 2023)
KRRSIGChange: 20231121050958 (Tue Nov 21 06:09:58 2023)
DSChange: 20231121050958 (Tue Nov 21 06:09:58 2023)
DNSKEYState: omnipresent
ZRRSIGState: omnipresent
KRRSIGState: omnipresent
DSState: rumoured
GoalState: omnipresent
```
Most recently, the old key's state changed at 2023-12-03 08:34:
```
; This is the state of key 44674, for rombobeorn.se.
Algorithm: 8
Length: 2048
Lifetime: 2678400
Predecessor: 26869
Successor: 58364
KSK: yes
ZSK: yes
Generated: 20231020040458 (Fri Oct 20 06:04:58 2023)
Published: 20231020040458 (Fri Oct 20 06:04:58 2023)
Active: 20231020060958 (Fri Oct 20 08:09:58 2023)
Retired: 20231120060958 (Mon Nov 20 07:09:58 2023)
Removed: 20231126071458 (Sun Nov 26 08:14:58 2023)
DSPublish: 20231020160958 (Fri Oct 20 18:09:58 2023)
PublishCDS: 20231020060958 (Fri Oct 20 08:09:58 2023)
DSPubCount: 1
DNSKEYChange: 20231020060958 (Fri Oct 20 08:09:58 2023)
ZRRSIGChange: 20231203073446 (Sun Dec 3 08:34:46 2023)
KRRSIGChange: 20231020060958 (Fri Oct 20 08:09:58 2023)
DSChange: 20231021180958 (Sat Oct 21 20:09:58 2023)
DNSKEYState: omnipresent
ZRRSIGState: hidden
KRRSIGState: omnipresent
DSState: omnipresent
GoalState: hidden
```
Other possibly useful state:
```
# rndc dnssec -status rombobeorn.se IN external
dnssec-policy: automatik
current time: Mon Dec 4 10:30:48 2023
key: 26869 (RSASHA256), CSK
published: no
key signing: no
zone signing: no
Key has been removed from the zone
- goal: hidden
- dnskey: hidden
- ds: hidden
- zone rrsig: hidden
- key rrsig: hidden
key: 44674 (RSASHA256), CSK
published: yes - since Fri Oct 20 06:04:58 2023
key signing: yes - since Fri Oct 20 06:04:58 2023
zone signing: no
Key is retired, will be removed on Sun Nov 26 08:14:58 2023
- goal: hidden
- dnskey: omnipresent
- ds: omnipresent
- zone rrsig: hidden
- key rrsig: omnipresent
key: 50640 (RSASHA256), ZSK
published: no
zone signing: no
Key has been removed from the zone
- goal: hidden
- dnskey: hidden
- ds: unretentive
- zone rrsig: hidden
- key rrsig: hidden
key: 48019 (RSASHA256), KSK
published: no
key signing: no
Key has been removed from the zone
- goal: hidden
- dnskey: hidden
- ds: hidden
- zone rrsig: hidden
- key rrsig: hidden
key: 58364 (RSASHA256), CSK
published: yes - since Mon Nov 20 05:04:58 2023
key signing: yes - since Mon Nov 20 05:04:58 2023
zone signing: yes - since Mon Nov 20 07:09:58 2023
Next rollover scheduled on Wed Dec 20 06:04:58 2023
- goal: omnipresent
- dnskey: omnipresent
- ds: rumoured
- zone rrsig: omnipresent
- key rrsig: omnipresent
```
```
# rndc zonestatus rombobeorn.se IN external
name: rombobeorn.se
type: primary
files: /var/lib/bind/db.rombobeorn.se.external
serial: 2023092684
nodes: 14
last loaded: Mon, 23 Oct 2023 21:53:52 GMT
secure: yes
inline signing: no
key maintenance: automatic
next key event: Mon, 04 Dec 2023 09:34:54 GMT
next resign node: rombobeorn.se/MX
next resign time: Mon, 04 Dec 2023 22:20:46 GMT
dynamic: yes
frozen: no
reconfigurable via modzone: no
```January 2024 (9.16.46, 9.16.46-S1, 9.18.22, 9.18.22-S1, 9.19.20) (❗RECALLED❗)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.org