ISC Open Source Projects issueshttps://gitlab.isc.org/groups/isc-projects/-/issues2020-11-02T11:32:00Zhttps://gitlab.isc.org/isc-projects/bind9/-/issues/2237BIND 9.16.8 cannot be cross-compiled2020-11-02T11:32:00ZMichał KępieńBIND 9.16.8 cannot be cross-compiledThis is a regression since BIND 9.16.7 that was inadvertently caused by
commit b8db702a0c7bc2310d2d134d099028bd250ed86c (part of !4211)
replacing `AC_LINK_IFELSE()` with `AC_RUN_IFELSE()` in `configure.ac`.
Cross-compiling BIND 9.16.8 is...This is a regression since BIND 9.16.7 that was inadvertently caused by
commit b8db702a0c7bc2310d2d134d099028bd250ed86c (part of !4211)
replacing `AC_LINK_IFELSE()` with `AC_RUN_IFELSE()` in `configure.ac`.
Cross-compiling BIND 9.16.8 is interrupted at `./configure` stage with
the following error message:
configure: error: cannot run test program while cross compiling
The problem does not affect the `main` branch because !4174 dropped
function wrapping from that branch altogether.November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)Michał KępieńMichał Kępieńhttps://gitlab.isc.org/isc-projects/bind9/-/issues/2235Add netmgr functions to support outgoing DNS queries2020-10-30T13:40:20ZOndřej SurýAdd netmgr functions to support outgoing DNS queriesNovember 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/2232bind9.xsl.h sometimes gets modified during the 'docs' CI job on v9_162023-12-12T11:33:18ZMichal Nowakbind9.xsl.h sometimes gets modified during the 'docs' CI job on v9_16On `v9_16` the `docs` CI jobs [sometimes](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1247836) (not the first time I saw it) modifies `bin/named/bind9.xsl.h` unexpectedly:
```
$ if test "$(git status --porcelain | grep -Ev '\?\?' | ...On `v9_16` the `docs` CI jobs [sometimes](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1247836) (not the first time I saw it) modifies `bin/named/bind9.xsl.h` unexpectedly:
```
$ if test "$(git status --porcelain | grep -Ev '\?\?' | grep -v -F -e aclocal.m4 -e configure -e ltmain.sh -e m4/ | wc -l)" -gt "0"; then git status --short; exit 1; fi
M aclocal.m4
M bin/named/bind9.xsl.h
M configure
M ltmain.sh
M m4/libtool.m4
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1
```BIND 9.17 Backburnerhttps://gitlab.isc.org/isc-projects/bind9/-/issues/2231error: undefined symbol: _Unwind_Backtrace on OpenBSD 6.82020-11-13T10:25:40ZMichal Nowakerror: undefined symbol: _Unwind_Backtrace on OpenBSD 6.8`v9_11` and `v9_16` (but not `main`) fail to build on OpenBSD 6.8 with Clang 10.0.1 (OpenBSD 6.7 with Clang 8.0.1 seems to be fine):
```
ld: error: undefined symbol: _Unwind_Backtrace
>>> referenced by backtrace.c:120
>>> ...`v9_11` and `v9_16` (but not `main`) fail to build on OpenBSD 6.8 with Clang 10.0.1 (OpenBSD 6.7 with Clang 8.0.1 seems to be fine):
```
ld: error: undefined symbol: _Unwind_Backtrace
>>> referenced by backtrace.c:120
>>> backtrace.o:(isc_backtrace_gettrace) in archive ../isc/libisc.a
ld: error: undefined symbol: _Unwind_GetIP
>>> referenced by backtrace.c:101
>>> backtrace.o:(btcallback) in archive ../isc/libisc.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error 1 in target 'sample-request'
ld: error: undefined symbol: _Unwind_Backtrace
>>> referenced by backtrace.c:120
>>> backtrace.o:(isc_backtrace_gettrace) in archive ../isc/libisc.a
ld: error: undefined symbol: _Unwind_GetIP
>>> referenced by backtrace.c:101
>>> backtrace.o:(btcallback) in archive ../isc/libisc.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error 1 in target 'resolve'
ld: error: undefined symbol: _Unwind_Backtrace
>>> referenced by backtrace.c:120
>>> backtrace.o:(isc_backtrace_gettrace) in archive ../isc/libisc.a
ld: error: undefined symbol: _Unwind_GetIP
>>> referenced by backtrace.c:101
>>> backtrace.o:(btcallback) in archive ../isc/libisc.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
Configuring with `--disable-backtrace` makes the build pass.November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)Michał KępieńMichał Kępieńhttps://gitlab.isc.org/isc-projects/bind9/-/issues/2228make depend fails with gcc: error: dnstap.pb-c.c: No such file or directory2020-11-04T21:22:11ZMichal Nowakmake depend fails with gcc: error: dnstap.pb-c.c: No such file or directoryOn `v9_11` `make depend` [prints](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1236814/raw):
```
$ make depend >/dev/null
gcc: error: dnstap.pb-c.c: No such file or directory
dnstap-read.c:51:10: fatal error: dns/dnstap.pb-c.h: No su...On `v9_11` `make depend` [prints](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1236814/raw):
```
$ make depend >/dev/null
gcc: error: dnstap.pb-c.c: No such file or directory
dnstap-read.c:51:10: fatal error: dns/dnstap.pb-c.h: No such file or directory
51 | #include <dns/dnstap.pb-c.h>
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
```
On `v9_11` and `v9_16` the command `make depend | grep "error:" && exit 1` from `.gitlab-ci.yml` won't work because the these errors go to stderr and we look for then on stdin. `make depend 2>&1 | grep "error:" && exit 1` should do.November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)https://gitlab.isc.org/isc-projects/bind9/-/issues/2227BIND 9.16.8 assertion failure2021-01-25T09:03:27ZAnand BuddhdevBIND 9.16.8 assertion failure### Summary
BIND crashed with an assertion failure in netmgr.c
### BIND version used
```
BIND 9.16.8 (Stable Release) <id:539f9f0>
running on Linux x86_64 3.10.0-957.21.3.el7.x86_64 #1 SMP Tue Jun 18 16:35:19 UTC 2019
built by make wi...### Summary
BIND crashed with an assertion failure in netmgr.c
### BIND version used
```
BIND 9.16.8 (Stable Release) <id:539f9f0>
running on Linux x86_64 3.10.0-957.21.3.el7.x86_64 #1 SMP Tue Jun 18 16:35:19 UTC 2019
built by make with '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc/named' '--disable-static' '--with-pic' '--without-python' '--with-libtool' '--without-lmdb' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -DISC_MEM_USE_INTERNAL_MALLOC=0 -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-z,relro ' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'
compiled by GCC 4.8.5 20150623 (Red Hat 4.8.5-39)
compiled with OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017
linked to OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017
compiled with libuv version: 1.40.0
linked to libuv version: 1.40.0
compiled with libxml2 version: 2.9.1
linked to libxml2 version: 20901
compiled with json-c version: 0.11
linked to json-c version: 0.11
compiled with zlib version: 1.2.7
linked to zlib version: 1.2.7
threads support is enabled
default paths:
named configuration: /etc/named/named.conf
rndc configuration: /etc/named/rndc.conf
DNSSEC root key: /etc/named/bind.keys
nsupdate session key: /var/run/named/session.key
named PID file: /var/run/named/named.pid
named lock file: /var/run/named/named.lock
```
### Steps to reproduce
I don't know.
### What is the current *bug* behavior?
BIND crashed and stopped serving queries.
### What is the expected *correct* behavior?
BIND should not crash.
### Relevant configuration files
```
acl "internal" {
key "main.ripe.net";
};
logging {
channel "default" {
file "/var/log/named/named.log";
severity info;
print-time yes;
print-category yes;
};
channel "ratelimit" {
file "/var/log/named/ratelimit.ringlog" versions 10 size 10485760;
print-time yes;
};
category "default" {
"default";
};
category "rate-limit" {
"ratelimit";
};
category "update" {
"null";
};
category "update-security" {
"null";
};
};
masters "hidden-main" {
IPv6-1 key "main.ripe.net";
IPv6-2 key "main.ripe.net";
IPv4-1 key "main.ripe.net";
IPv4-2 key "main.ripe.net";
};
options {
answer-cookie no;
directory "/var/named";
keep-response-order {
"any";
};
listen-on {
127.0.0.1/32;
IPv4/32;
193.0.14.129/32;
193.0.15.129/32;
};
listen-on-v6 {
::1/128;
IPv6/128;
2001:7fd::1/128;
2001:7fd:15::1/128;
};
server-id hostname;
tcp-clients 1000;
version "9.16";
dnssec-validation no;
minimal-responses yes;
recursion no;
allow-transfer {
"internal";
};
max-journal-size 10485760;
notify explicit;
zero-no-soa-ttl no;
zone-statistics none;
};
key "main.ripe.net" {
algorithm "hmac-sha256";
secret "????????????????????????????????????????????";
};
zone "." {
type slave;
file ".zone";
masters {
"hidden-main";
};
allow-transfer {
"any";
};
};
zone "arpa." {
type slave;
file "arpa.zone";
masters {
"hidden-main";
};
allow-transfer {
"any";
};
};
zone "root-servers.net." {
type slave;
file "root-servers.net.zone";
masters {
"hidden-main";
};
allow-transfer {
"any";
};
};
```
### Relevant logs and/or screenshots
```
22-Oct-2020 12:45:30.655 general: netmgr.c:1176: REQUIRE(((__builtin_expect(!!((handle) != ((void *)0)), 1) && __builtin_expect(!!(((const isc__magic_t *)(handle))->magic == ((('N') << 24 | ('M') << 16 | ('H') << 8 | ('D')))), 1)) && __atomic_load_n(&(handle)->references, memory_order_seq_cst) > 0)) failed, back trace
22-Oct-2020 12:45:30.655 general: #0 0x42b597 in ??
22-Oct-2020 12:45:30.655 general: #1 0x7f251d7ea2da in ??
22-Oct-2020 12:45:30.655 general: #2 0x7f251d801750 in ??
22-Oct-2020 12:45:30.655 general: #3 0x7f251d8088bc in ??
22-Oct-2020 12:45:30.655 general: #4 0x7f251ee402a7 in ??
22-Oct-2020 12:45:30.655 general: #5 0x7f251ee41c6a in ??
22-Oct-2020 12:45:30.655 general: #6 0x7f251ee50d2c in ??
22-Oct-2020 12:45:30.655 general: #7 0x7f251ee586ee in ??
22-Oct-2020 12:45:30.655 general: #8 0x7f251ee59305 in ??
22-Oct-2020 12:45:30.655 general: #9 0x7f251ee60a6a in ??
22-Oct-2020 12:45:30.655 general: #10 0x7f251ee5d0d1 in ??
22-Oct-2020 12:45:30.655 general: #11 0x7f251ee5f6aa in ??
22-Oct-2020 12:45:30.655 general: #12 0x7f251ee5fd36 in ??
22-Oct-2020 12:45:30.655 general: #13 0x7f251ee608ea in ??
22-Oct-2020 12:45:30.655 general: #14 0x7f251ee44fb0 in ??
22-Oct-2020 12:45:30.655 general: #15 0x7f251d807bac in ??
22-Oct-2020 12:45:30.655 general: #16 0x7f251d807feb in ??
22-Oct-2020 12:45:30.655 general: #17 0x7f251d8042a4 in ??
22-Oct-2020 12:45:30.655 general: #18 0x7f251cc2e164 in ??
22-Oct-2020 12:45:30.655 general: #19 0x7f251cc2ee7c in ??
22-Oct-2020 12:45:30.655 general: #20 0x7f251cc348c3 in ??
22-Oct-2020 12:45:30.655 general: #21 0x7f251cc240d0 in ??
22-Oct-2020 12:45:30.655 general: #22 0x7f251d8037a9 in ??
22-Oct-2020 12:45:30.655 general: #23 0x7f251c396ea5 in ??
22-Oct-2020 12:45:30.655 general: #24 0x7f251c0bf8dd in ??
22-Oct-2020 12:45:30.655 general: exiting (due to assertion failure)
```
### Possible fixes
I don't know.November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)https://gitlab.isc.org/isc-projects/bind9/-/issues/2226TSAN error in dispatch.c accessing disp->portentry2020-10-30T10:33:58ZMark AndrewsTSAN error in dispatch.c accessing disp->portentryJob [#1237048](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1237048) failed for 33ea90f76711e400b98078cc8425d4dda4366609:
```
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by thread T1 (mutexes: write M1):...Job [#1237048](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1237048) failed for 33ea90f76711e400b98078cc8425d4dda4366609:
```
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by thread T1 (mutexes: write M1):
#0 deref_portentry lib/dns/dispatch.c:630
#1 deactivate_dispsocket lib/dns/dispatch.c:861
#2 udp_recv lib/dns/dispatch.c:1105
#3 udp_exrecv lib/dns/dispatch.c:1028
#4 dispatch lib/isc/task.c:1152
#5 run lib/isc/task.c:1344
#6 <null> <null>
Previous read of size 8 at 0x000000000001 by thread T2 (mutexes: write M1, write M2):
#0 socket_search lib/dns/dispatch.c:661
#1 get_dispsocket lib/dns/dispatch.c:744
#2 dns_dispatch_addresponse lib/dns/dispatch.c:3120
#3 resquery_send lib/dns/resolver.c:2467
#4 fctx_query lib/dns/resolver.c:2217
#5 fctx_try lib/dns/resolver.c:4245
#6 fctx_timeout lib/dns/resolver.c:4570
#7 dispatch lib/isc/task.c:1152
#8 run lib/isc/task.c:1344
#9 <null> <null>
Location is heap block of size 249 at 0x000000000013 allocated by thread T2:
#0 malloc <null>
#1 default_memalloc lib/isc/mem.c:713
#2 mem_get lib/isc/mem.c:622
#3 isc__mempool_get lib/isc/mem.c:1775
#4 get_dispsocket lib/dns/dispatch.c:712
#5 dns_dispatch_addresponse lib/dns/dispatch.c:3120
#6 resquery_send lib/dns/resolver.c:2467
#7 fctx_query lib/dns/resolver.c:2217
#8 fctx_try lib/dns/resolver.c:4245
#9 fctx_start lib/dns/resolver.c:4761
#10 dispatch lib/isc/task.c:1152
#11 run lib/isc/task.c:1344
#12 <null> <null>
Mutex M1 (0x000000000019) created at:
#0 pthread_mutex_init <null>
#1 isc__mutex_init pthreads/mutex.c:288
#2 dispatch_allocate lib/dns/dispatch.c:2358
#3 dispatch_createudp lib/dns/dispatch.c:2855
#4 dns_dispatchset_create lib/dns/dispatch.c:3666
#5 dns_resolver_create lib/dns/resolver.c:10190
#6 dns_view_createresolver lib/dns/view.c:811
#7 configure_view bin/named/server.c:4571
#8 load_configuration bin/named/server.c:9110
#9 loadconfig bin/named/server.c:10310
#10 named_server_reconfigcommand bin/named/server.c:10693
#11 named_control_docommand bin/named/control.c:250
#12 control_command bin/named/controlconf.c:392
#13 dispatch lib/isc/task.c:1152
#14 run lib/isc/task.c:1344
#15 <null> <null>
Mutex M2 is already destroyed.
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create pthreads/thread.c:73
#2 isc_taskmgr_create lib/isc/task.c:1434
#3 create_managers bin/named/main.c:915
#4 setup bin/named/main.c:1223
#5 main bin/named/main.c:1523
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create pthreads/thread.c:73
#2 isc_taskmgr_create lib/isc/task.c:1434
#3 create_managers bin/named/main.c:915
#4 setup bin/named/main.c:1223
#5 main bin/named/main.c:1523
SUMMARY: ThreadSanitizer: data race lib/dns/dispatch.c:630 in deref_portentry
```November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)https://gitlab.isc.org/isc-projects/bind9/-/issues/2224Fixup core back traces.2020-12-01T08:43:37ZMark AndrewsFixup core back traces.The back traces fail to identify which core they are from.
The back traces can be over written by another backtrace for the same executable.
Binary not correctly identified (subject of another MR).The back traces fail to identify which core they are from.
The back traces can be over written by another backtrace for the same executable.
Binary not correctly identified (subject of another MR).December 2020 (9.11.26, 9.11.26-S1, 9.16.10, 9.16.10-S1, 9.17.8)https://gitlab.isc.org/isc-projects/bind9/-/issues/2221Create unit tests for netmgr2020-12-03T10:57:40ZOndřej SurýCreate unit tests for netmgrDecember 2020 (9.11.26, 9.11.26-S1, 9.16.10, 9.16.10-S1, 9.17.8)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/2220Update OS images (Fedora 33, OpenBSD 6.8, FreeBSD 12.2)2020-11-13T11:17:10ZMichal NowakUpdate OS images (Fedora 33, OpenBSD 6.8, FreeBSD 12.2)Update OS images, all are either already out or out in a few weeks.
- [x] OpenBSD 6.8: https://gitlab.isc.org/isc-projects/images/-/merge_requests/81
- [x] Fedora 33: https://gitlab.isc.org/isc-projects/images/-/merge_requests/82
- [x] F...Update OS images, all are either already out or out in a few weeks.
- [x] OpenBSD 6.8: https://gitlab.isc.org/isc-projects/images/-/merge_requests/81
- [x] Fedora 33: https://gitlab.isc.org/isc-projects/images/-/merge_requests/82
- [x] FreeBSD 12.2: https://gitlab.isc.org/isc-projects/images/-/merge_requests/83
- [x] Also there's Ubuntu 20.10 (Groovy Gorilla) a non-LTS release which we don't intend to add to our OS images, but should probably build and test on as a one-off sanity test.November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)Michal NowakMichal Nowakhttps://gitlab.isc.org/isc-projects/stork/-/issues/430UI tooltips misaligned2021-01-28T10:43:36ZTomek MrugalskiUI tooltips misalignedAs reported here https://gitlab.isc.org/isc-projects/stork/-/issues/426#note_169320 by @godfryd:
![image](/uploads/6ba0d52a3875a0d8261d90788910060b/image.png)
Help box text is centered instead of aligned to the left.
This is probably d...As reported here https://gitlab.isc.org/isc-projects/stork/-/issues/426#note_169320 by @godfryd:
![image](/uploads/6ba0d52a3875a0d8261d90788910060b/image.png)
Help box text is centered instead of aligned to the left.
This is probably due to the fact that question mark icon is located in the table header which is also centered. The problem applies to other boxes that have its question mark icon in table headers.0.15https://gitlab.isc.org/isc-projects/bind9/-/issues/2218Ensure use of "echo_i" where possible in system tests2020-10-30T10:23:25ZMichal NowakEnsure use of "echo_i" where possible in system testsInstead of using `echo_i`, some tests print output via `echo "I:...`, which in many instances misses test name, e.g. `tsiggss`:
```
S:tsiggss:2020-10-15T10:16:58+0200
T:tsiggss:1:A
A:tsiggss:System test tsiggss
I:tsiggss:PORTS:12408,1240...Instead of using `echo_i`, some tests print output via `echo "I:...`, which in many instances misses test name, e.g. `tsiggss`:
```
S:tsiggss:2020-10-15T10:16:58+0200
T:tsiggss:1:A
A:tsiggss:System test tsiggss
I:tsiggss:PORTS:12408,12409,12410,12411,12412,12413,12414,12415,12416,12417
I:tsiggss:starting servers
I:testing updates to testdc1 as administrator (1)
```
I am not sure which branch was that but recently I saw a file with all `*.log` files merged (and sorted?) and it wasn't clear to which a "failed" info line belong.November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)Michal NowakMichal Nowakhttps://gitlab.isc.org/isc-projects/bind9/-/issues/2215DNS_ZONEFLAG_NOIXFR is misnamed2020-10-30T10:31:56ZBrian ConryDNS_ZONEFLAG_NOIXFR is misnamedAll of the other zone flags are prefixed `DNS_ZONEFLG` instead of `DNS_ZONEFLAG`All of the other zone flags are prefixed `DNS_ZONEFLG` instead of `DNS_ZONEFLAG`November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)https://gitlab.isc.org/isc-projects/bind9/-/issues/2211tsan error previous_closest_nsec(dns_rbt_findnode) vs subtractrdataset2020-11-12T14:04:00ZMark Andrewstsan error previous_closest_nsec(dns_rbt_findnode) vs subtractrdatasetJob [#1217843](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1217843) failed for 834f0f5a1a55a6e8cd08dde6a481332a43cf8c0c:
```
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1 (mutexes: read M1, re...Job [#1217843](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1217843) failed for 834f0f5a1a55a6e8cd08dde6a481332a43cf8c0c:
```
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1 (mutexes: read M1, read M2):
#0 dns_rbt_findnode lib/dns/rbt.c:1708
#1 previous_closest_nsec lib/dns/rbtdb.c:3760
#2 find_closest_nsec lib/dns/rbtdb.c:3942
#3 zone_find lib/dns/rbtdb.c:4091
#4 dns_db_findext lib/dns/db.c:536
#5 query_lookup lib/ns/query.c:5582
#6 ns__query_start lib/ns/query.c:5505
#7 query_setup lib/ns/query.c:5229
#8 ns_query_start lib/ns/query.c:11380
#9 ns__client_request lib/ns/client.c:2166
#10 processbuffer netmgr/tcpdns.c:230
#11 dnslisten_readcb netmgr/tcpdns.c:309
#12 read_cb netmgr/tcp.c:832
#13 <null> <null>
#14 <null> <null>
Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M3):
#0 subtractrdataset lib/dns/rbtdb.c:7133
#1 dns_db_subtractrdataset lib/dns/db.c:742
#2 diff_apply lib/dns/diff.c:368
#3 dns_diff_apply lib/dns/diff.c:459
#4 do_one_tuple lib/dns/update.c:247
#5 update_one_rr lib/dns/update.c:275
#6 delete_if_action lib/dns/update.c:689
#7 foreach_rr lib/dns/update.c:471
#8 delete_if lib/dns/update.c:716
#9 dns_update_signaturesinc lib/dns/update.c:1948
#10 receive_secure_serial lib/dns/zone.c:15637
#11 dispatch lib/isc/task.c:1152
#12 run lib/isc/task.c:1344
#13 <null> <null>
Location is heap block of size 130 at 0x000000000028 allocated by thread T3:
#0 malloc <null>
#1 default_memalloc lib/isc/mem.c:713
#2 mem_get lib/isc/mem.c:622
#3 mem_allocateunlocked lib/isc/mem.c:1268
#4 isc___mem_allocate lib/isc/mem.c:1288
#5 isc__mem_allocate lib/isc/mem.c:2453
#6 isc___mem_get lib/isc/mem.c:1037
#7 isc__mem_get lib/isc/mem.c:2432
#8 create_node lib/dns/rbt.c:2239
#9 dns_rbt_addnode lib/dns/rbt.c:1202
#10 dns_rbtdb_create lib/dns/rbtdb.c:8668
#11 dns_db_create lib/dns/db.c:118
#12 receive_secure_db lib/dns/zone.c:16154
#13 dispatch lib/isc/task.c:1152
#14 run lib/isc/task.c:1344
#15 <null> <null>
Mutex M1 (0x000000000040) created at:
#0 pthread_rwlock_init <null>
#1 isc_rwlock_init lib/isc/rwlock.c:39
#2 dns_rbtdb_create lib/dns/rbtdb.c:8527
#3 dns_db_create lib/dns/db.c:118
#4 receive_secure_db lib/dns/zone.c:16154
#5 dispatch lib/isc/task.c:1152
#6 run lib/isc/task.c:1344
#7 <null> <null>
Mutex M2 (0x000000000044) created at:
#0 pthread_rwlock_init <null>
#1 isc_rwlock_init lib/isc/rwlock.c:39
#2 dns_rbtdb_create lib/dns/rbtdb.c:8600
#3 dns_db_create lib/dns/db.c:118
#4 receive_secure_db lib/dns/zone.c:16154
#5 dispatch lib/isc/task.c:1152
#6 run lib/isc/task.c:1344
#7 <null> <null>
Mutex M3 (0x000000000046) created at:
#0 pthread_rwlock_init <null>
#1 isc_rwlock_init lib/isc/rwlock.c:39
#2 dns_rbtdb_create lib/dns/rbtdb.c:8600
#3 dns_db_create lib/dns/db.c:118
#4 receive_secure_db lib/dns/zone.c:16154
#5 dispatch lib/isc/task.c:1152
#6 run lib/isc/task.c:1344
#7 <null> <null>
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create pthreads/thread.c:73
#2 isc_nm_start netmgr/netmgr.c:232
#3 create_managers bin/named/main.c:909
#4 setup bin/named/main.c:1223
#5 main bin/named/main.c:1523
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create pthreads/thread.c:73
#2 isc_taskmgr_create lib/isc/task.c:1434
#3 create_managers bin/named/main.c:915
#4 setup bin/named/main.c:1223
#5 main bin/named/main.c:1523
Thread T3 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create pthreads/thread.c:73
#2 isc_taskmgr_create lib/isc/task.c:1434
#3 create_managers bin/named/main.c:915
#4 setup bin/named/main.c:1223
#5 main bin/named/main.c:1523
SUMMARY: ThreadSanitizer: data race lib/dns/rbt.c:1708 in dns_rbt_findnode
```November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)https://gitlab.isc.org/isc-projects/bind9/-/issues/2209TSAN error bin/named/controlconf.c related.2023-11-23T08:14:30ZMark AndrewsTSAN error bin/named/controlconf.c related.Job [#1213042](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1213042) failed for ced5041de73f16dbadf1acacbf8c20659efcb6a6:
```
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by thread T1:
#0 isc_nmhandle...Job [#1213042](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1213042) failed for ced5041de73f16dbadf1acacbf8c20659efcb6a6:
```
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by thread T1:
#0 isc_nmhandle_detach lib/isc/netmgr/netmgr.c:1258:15
#1 control_command bin/named/controlconf.c:388:3
#2 dispatch lib/isc/task.c:1152:7
#3 run lib/isc/task.c:1344:2
Previous read of size 8 at 0x000000000001 by thread T2:
#0 isc_nm_pauseread lib/isc/netmgr/netmgr.c:1449:33
#1 recv_data lib/isccc/ccmsg.c:109:2
#2 isc__nm_tcp_shutdown lib/isc/netmgr/tcp.c:1157:4
#3 shutdown_walk_cb lib/isc/netmgr/netmgr.c:1515:3
#4 uv_walk <null>
#5 process_queue lib/isc/netmgr/netmgr.c:659:4
#6 process_normal_queue lib/isc/netmgr/netmgr.c:582:10
#7 process_queues lib/isc/netmgr/netmgr.c:590:8
#8 async_cb lib/isc/netmgr/netmgr.c:548:2
#9 <null> <null>
Location is heap block of size 569 at 0x000000000016 allocated by thread T2:
#0 malloc <null>
#1 default_memalloc lib/isc/mem.c:713:8
#2 mem_get lib/isc/mem.c:622:8
#3 isc___mem_get lib/isc/mem.c:1044:9
#4 isc__mem_get lib/isc/mem.c:2432:10
#5 alloc_handle lib/isc/netmgr/netmgr.c:1076:3
#6 isc__nmhandle_get lib/isc/netmgr/netmgr.c:1100:12
#7 isc__nm_async_tcpchildaccept lib/isc/netmgr/tcp.c:486:11
#8 process_queue lib/isc/netmgr/netmgr.c:624:4
#9 process_normal_queue lib/isc/netmgr/netmgr.c:582:10
#10 process_queues lib/isc/netmgr/netmgr.c:590:8
#11 async_cb lib/isc/netmgr/netmgr.c:548:2
#12 <null> <null>
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_nm_start lib/isc/netmgr/netmgr.c:232:3
#3 create_managers bin/named/main.c:909:15
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: data race lib/isc/netmgr/netmgr.c:1258:15 in isc_nmhandle_detach
```November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)https://gitlab.isc.org/isc-projects/bind9/-/issues/2208TCP4RECVERR counter mis-counting on 9.16+?2020-10-30T11:07:28ZMichael McNallyTCP4RECVERR counter mis-counting on 9.16+?Via (Support Ticket #17204](https://support.isc.org/Ticket/Display.html?id=17204), Dave F. tells us of a problem with the TCP4RECVERR counter that sounds as though we are probably mis-counting in releases that use the new netmgr.
> We'r...Via (Support Ticket #17204](https://support.isc.org/Ticket/Display.html?id=17204), Dave F. tells us of a problem with the TCP4RECVERR counter that sounds as though we are probably mis-counting in releases that use the new netmgr.
> We're attempting upgrade from 9.14.12 to 9.16.[5,6,7]. Upon upgrade we started getting internal alarms firing because the Bind internal stat TCP4RecvErr increases on every DNS TCP query we send. The DNS queries get valid DNS responses, so if we hadn't been alarming on the stat we never would have noticed an immediate issue.
>
> ...
>
> We saw this problem on 9.16.5 and then repro'd it on .6 and .7.
>
>
> To help with the debug, I am including some stat blocks.
>
> This is the initial state of the stats before we issue the queries:
```
{
"json-stats-version":"1.5",
"boot-time":"2020-10-08T19:44:32.194Z",
"config-time":"2020-10-08T20:47:37.818Z",
"current-time":"2020-10-08T20:55:05.270Z",
"version":"9.16.7",
"sockstats":{
"UDP4Open":2,
"TCP4Open":3,
"RawOpen":1,
"TCP4Close":243,
"TCP4Accept":244,
"TCP4RecvErr":88,
"UDP4Active":3,
"TCP4Active":4,
"RawActive":1
},
```
> We then run two TCP queries using the same version of dig 9.16.7 built with this version of bind under test.
```
bash-4.2$ dig @127.0.0.1 localhost a +tcp
; <<>> DiG 9.16.7 <<>> @127.0.0.1 localhost a +tcp
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 4021
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: ba9ced48a62b6061010000005f7f7cc12056d6b3fad00a0c (good)
;; QUESTION SECTION:
;localhost. IN A
;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Oct 08 20:55:29 UTC 2020
;; MSG SIZE rcvd: 66
bash-4.2$ dig @127.0.0.1 <redacted> TXT +tcp
; <<>> DiG 9.16.7 <<>> @127.0.0.1 <redacted> TXT +tcp
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28547
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: bca3ab06023e0f15010000005f7f7cc7c9512f87cfdff056 (good)
;; QUESTION SECTION:
;<redacted>. IN TXT
;; ANSWER SECTION:
<redacted>. 3600 IN TXT "OK"
;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu Oct 08 20:55:35 UTC 2020
;; MSG SIZE rcvd: 101
```
> Note that we have a REFUSED response and an NOERROR response, but both valid DNS responses with no sign of TCP error.
>
> After running these queries the stat block looks like:
```
bash-4.2$ curl http://localhost:8053/json/v1/net
{
"json-stats-version":"1.5",
"boot-time":"2020-10-08T19:44:32.194Z",
"config-time":"2020-10-08T20:47:37.818Z",
"current-time":"2020-10-08T20:55:37.681Z",
"version":"9.16.7",
"sockstats":{
"UDP4Open":2,
"TCP4Open":3,
"RawOpen":1,
"TCP4Close":246,
"TCP4Accept":247,
"TCP4RecvErr":90,
"UDP4Active":3,
"TCP4Active":4,
"RawActive":1
},
```
> Note that TCP4RecvErr has increased by 2, corresponding to the 2 queries. This is the concern.
>
> TCP4Close and TCP4Accept both increase by 3 from the baseline due to the 2 queries plus the curl request to obtain the stats. This seems to be normal.
>
> Please let me know if I can provide any more information.
> We'd like to understand if this is a known issue and if there are any other concerns (memory leak, socket leak, performance issue) associated with the error.November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/2204Investigate pipeline system test failure2020-10-08T08:23:05ZMark AndrewsInvestigate pipeline system test failureJob [#1209225](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1209225) failed for 37e4eb53c91fb2cb32619c222239c75f41ae1f3b:
```
FAIL: pipelined
7719===============
7720S:pipelined:2020-10-07T23:20:20+0000
7721T:pipelined:1:A
7722A:pip...Job [#1209225](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1209225) failed for 37e4eb53c91fb2cb32619c222239c75f41ae1f3b:
```
FAIL: pipelined
7719===============
7720S:pipelined:2020-10-07T23:20:20+0000
7721T:pipelined:1:A
7722A:pipelined:System test pipelined
7723I:pipelined:PORTS:8339,8340,8341,8342,8343,8344,8345,8346,8347,8348
7724I:pipelined:starting servers
7725I:pipelined:check pipelined TCP queries
7726I:pipelined:check pipelined TCP queries using mdig
7727response failed with unexpected error
77283d2
7729< b.examplea. 10.0.1.2
77307d5
7731< d.examplea. 10.0.1.4
7732I:pipelined:diff sorted failed
7733I:pipelined:failed
7734I:pipelined:check keep-response-order
7735rndc: connect failed: 10.53.0.4#8348: connection refused
7736unix/socket.c:4772: connect(10.53.0.4#8339) 103/Software caused connection abort
7737I:dns_request_create: unexpected error
77381,8d0
7739< e.examplea. 10.0.1.5
7740< e.exampleb. 10.0.2.5
7741< f.examplea. 10.0.1.6
7742< f.exampleb. 10.0.2.6
7743< g.examplea. 10.0.1.7
7744< g.exampleb. 10.0.2.7
7745< h.examplea. 10.0.1.8
7746< h.exampleb. 10.0.2.8
7747I:pipelined:failed
7748I:pipelined:check keep-response-order using mdig
7749rndc: connect failed: 10.53.0.4#8348: connection refused
7750unix/socket.c:4772: connect(10.53.0.4#8339) 103/Software caused connection abort
7751mdig: dns_request_createvia4 failed with unexpected error
77521,8d0
7753< e.examplea. 10.0.1.5
7754< e.exampleb. 10.0.2.5
7755< f.examplea. 10.0.1.6
7756< f.exampleb. 10.0.2.6
7757< g.examplea. 10.0.1.7
7758< g.exampleb. 10.0.2.7
7759< h.examplea. 10.0.1.8
7760< h.exampleb. 10.0.2.8
7761I:pipelined:failed
7762I:pipelined:check mdig -4 -6
7763rndc: connect failed: 10.53.0.4#8348: connection refused
7764I:pipelined:check mdig -4 with an IPv6 server address
7765I:pipelined:exit status: 3
7766I:pipelined:stopping servers
7767I:pipelined:ns4 died before a SIGTERM was sent
7768I:pipelined:stopping servers failed
7769I:pipelined:1 assertion failure(s) found
7770R:pipelined:FAIL
7771E:pipelined:2020-10-07T23:20:43+0000
7772FAIL pipelined (exit status: 1)
```October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.16.8-S1, 9.17.6)https://gitlab.isc.org/isc-projects/bind9/-/issues/2198CID 309658: Null pointer dereferences (REVERSE_INULL) in lib/dns/zone.c2020-10-07T08:59:06ZMichal NowakCID 309658: Null pointer dereferences (REVERSE_INULL) in lib/dns/zone.cCoverity Scan identified null pointer dereference on [`main`](https://scan8.coverity.com/reports.htm#v38342/p12579/fileInstanceId=36270434&defectInstanceId=11118673&mergedDefectId=309658) and later on [`v9_16`](https://scan8.coverity.com...Coverity Scan identified null pointer dereference on [`main`](https://scan8.coverity.com/reports.htm#v38342/p12579/fileInstanceId=36270434&defectInstanceId=11118673&mergedDefectId=309658) and later on [`v9_16`](https://scan8.coverity.com/reports.htm#v38342/p12582/fileInstanceId=36271991&defectInstanceId=11118776&mergedDefectId=309658):
```
*** CID 309658: Null pointer dereferences (REVERSE_INULL)
/lib/dns/zone.c: 13461 in create_query()
13455 if (qname != NULL) {
13456 dns_message_puttempname(message, &qname);
13457 }
13458 if (qrdataset != NULL) {
13459 dns_message_puttemprdataset(message, &qrdataset);
13460 }
>>> CID 309658: Null pointer dereferences (REVERSE_INULL)
>>> Null-checking "message" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
13461 if (message != NULL) {
13462 dns_message_detach(&message);
13463 }
13464 return (result);
13465 }
13466
```
Adding to %"October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.17.6)" milestone as this slipped-in this milestone.October 2020 (9.11.24, 9.11.24-S1, 9.16.8, 9.16.8-S1, 9.17.6)https://gitlab.isc.org/isc-projects/bind9/-/issues/2195FreeBSD dnstap system test failure2021-02-23T14:20:11ZDiego dos Santos FronzaFreeBSD dnstap system test failurednstap system test on FreeBSD may fail due to an attempt in reading a not yet flushed dnstap file captured by fstrm_capture tool.dnstap system test on FreeBSD may fail due to an attempt in reading a not yet flushed dnstap file captured by fstrm_capture tool.November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)Diego dos Santos FronzaDiego dos Santos Fronzahttps://gitlab.isc.org/isc-projects/bind9/-/issues/2193TSAN errors in v9_16 to be investigated.2020-10-02T01:43:18ZMark AndrewsTSAN errors in v9_16 to be investigated.Job [#1192145](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1192145) failed for fc3cab22a44cff8458cf39865836c7e5d883cbed:
```
I:System test result summary:
I: 2 FAIL
I: 94 PASS
I: 3 SKIPPED
I:The following system tests...Job [#1192145](https://gitlab.isc.org/isc-projects/bind9/-/jobs/1192145) failed for fc3cab22a44cff8458cf39865836c7e5d883cbed:
```
I:System test result summary:
I: 2 FAIL
I: 94 PASS
I: 3 SKIPPED
I:The following system tests failed:
I: nsupdate
I: tcp
I:ThreadSanitizer reported issues for the following system tests:
I: tcp
```
Races over access to `sock->rcb.recv` and `sock->rcbarg`. `sock->rcbarg` in this case.
`sock->rcbarg` is set to NULL before calling `isc_nm_stoplistening`. Additionally we
need to wait for isc_nm_stoplistening to be processed when it is sent via the event
queue.
```
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1:
#0 processbuffer lib/isc/netmgr/tcpdns.c:185:15
#1 resume_processing lib/isc/netmgr/tcpdns.c:421:12
#2 isc_nmhandle_unref lib/isc/netmgr/netmgr.c:1173:4
#3 isc__nm_uvreq_put lib/isc/netmgr/netmgr.c:1288:3
#4 tcpdnssend_cb lib/isc/netmgr/tcpdns.c:449:2
#5 tcp_send_cb lib/isc/netmgr/tcp.c:892:2
#6 <null> <null>
Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M1):
#0 memset <null>
#1 isc__nm_tcpdns_stoplistening lib/isc/netmgr/tcpdns.c:330:15
#2 isc_nm_stoplistening lib/isc/netmgr/netmgr.c:1358:3
#3 ns_interface_shutdown lib/ns/interfacemgr.c:564:3
#4 purge_old_interfaces lib/ns/interfacemgr.c:658:4
#5 ns_interfacemgr_shutdown lib/ns/interfacemgr.c:386:2
#6 shutdown_server bin/named/./server.c:9851:2
#7 dispatch lib/isc/task.c:1152:7
#8 run lib/isc/task.c:1344:2
```November 2020 (9.11.25, 9.11.25-S1, 9.16.9, 9.16.9-S1, 9.17.7)