BIND issueshttps://gitlab.isc.org/isc-projects/bind9/-/issues2020-02-07T12:58:51Zhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1556Release Checklist for BIND 9.11.15, BIND 9.11.15-S1, BIND 9.14.10, BIND 9.15.82020-02-07T12:58:51ZMichał KępieńRelease Checklist for BIND 9.11.15, BIND 9.11.15-S1, BIND 9.14.10, BIND 9.15.8## Release Schedule
**Tagging Deadline:** Wednesday, January 15th, 2020
**Public Release:** Wednesday, January 22nd, 2020
## Release Checklist
## 2 Working Days Before the Tagging Deadline
- [x] ***(QA)*** Check whether all issues ...## Release Schedule
**Tagging Deadline:** Wednesday, January 15th, 2020
**Public Release:** Wednesday, January 22nd, 2020
## Release Checklist
## 2 Working Days Before the Tagging Deadline
- [x] ***(QA)*** Check whether all issues assigned to the release milestone are resolved[^1].
- [x] ***(QA)*** Ensure that there are no outstanding merge requests in the private repository[^1] (Subscription Edition only).
- [x] ***(QA)*** Ensure all merge requests marked for backporting have been indeed backported.
## Before the Tagging Deadline
- [x] ***(QA)*** Inform Support/Marketing of impending release (and give estimated release dates).
- [x] ***(QA)*** Check Perflab to ensure there has been no unexplained drop in performance for the versions being released.
- [x] ***(SwEng)*** Update API files for libraries with new version information.
- [x] ***(SwEng)*** Change software version and library versions in `configure.ac` (new major release only).
- [x] ***(SwEng)*** Rebuild `configure` using Autoconf on `docs.isc.org`.
- [x] ***(SwEng)*** Update `CHANGES`.
- [x] ***(SwEng)*** Update `CHANGES.SE` (Subscription Edition only).
- [x] ***(SwEng)*** Update `README.md`.
- [x] ***(SwEng)*** Update `version`.
- [x] ***(SwEng)*** Build documentation on `docs.isc.org`.
- [x] ***(QA)*** Check that all the above steps were performed correctly.
- [x] ***(QA)*** Check that the contents of release notes match the merge requests comprising the releases.
- [x] ***(QA)*** Check that the formatting is correct for text, PDF, and HTML versions of release notes.
- [x] ***(SwEng)*** Tag the releases[^2]. (Tags may only be pushed to the public repository for releases which are *not* security releases.)
- [x] ***(SwEng)*** If this is the first tag for a release (e.g. beta), create a release branch named `release_v9_X_Y` to allow development to continue on the maintenance branch whilst release engineering continues.
## Before the ASN Deadline (for ASN Releases) or the Public Release Date (for Regular Releases)
- [x] ***(QA)*** Verify GitLab CI results for the tags created and prepare a QA report for the releases to be published.
- [x] ***(QA)*** Request signatures for the tarballs, providing their location and checksums.
- [x] ***(Signers)*** Validate tarball checksums, sign tarballs, and upload signatures.
- [x] ***(QA)*** Verify tarball signatures and check tarball checksums again.
- [x] ***(Support)*** Pre-publish ASN and/or Subscription Edition tarballs so that packages can be built.
- [x] ***(QA)*** Build and test ASN and/or Subscription Edition packages.
- [x] ***(QA)*** Notify Support that the releases have been prepared.
- [x] ***(Support)*** Send out ASNs (if applicable).
## On the Day of Public Release
- [x] ***(Support)*** Wait for clearance from Security Officer to proceed with the public release (if applicable).
- [x] ***(Support)*** Place tarballs in public location on FTP site.
- [x] ***(Support)*** Publish links to downloads on ISC website.
- [x] ***(Support)*** Write release email to *bind-announce*.
- [x] ***(Support)*** Write email to *bind-users* (if a major release).
- [x] ***(Support)*** Update tickets in case of waiting support customers.
- [x] ***(QA)*** Build and test any outstanding private packages.
- [x] ***(QA)*** Build public packages (`*.deb`, RPMs).
- [x] ***(QA)*** Inform Marketing of the release.
- [x] ***(QA)*** Update the internal [BIND release dates wiki page](https://wiki.isc.org/bin/view/Main/BindReleaseDates) when public announcement has been made.
- [x] ***(Marketing)*** Post short note to Twitter.
- [x] ***(Marketing)*** Update [Wikipedia entry for BIND](https://en.wikipedia.org/wiki/BIND).
- [x] ***(Marketing)*** Write blog article (if a major release).
- [x] ***(QA)*** Ensure all new tags are annotated and signed.
- [x] ***(SwEng)*** Push tags for the published releases to the public repository.
- [x] ***(SwEng)*** Merge the automatically prepared `prep 9.X.Y` commit which updates `version` and documentation on the release branch into the relevant maintenance branch (`v9_X`).
[^1]: If not, use the time remaining until the tagging deadline to ensure all outstanding issues are either resolved or moved to a different milestone.
[^2]: Preferred command line: `git tag -u <DEVELOPER_KEYID> -a -s -m "BIND 9.X.Y[alphatag]" v9_X_Y[alphatag]`, where `[alphatag]` is an optional string such as `b1`, `rc1`, etc.January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Michał KępieńMichał Kępień2020-01-22https://gitlab.isc.org/isc-projects/bind9/-/issues/1555Code with pthread specific semantics is not portable2020-01-13T08:54:56ZOndřej SurýCode with pthread specific semantics is not portableWhen working on C11 thread library support, I found two cases where we use pthread specific semantics:
* Use of `void *` as a return type
* Use of `PTHREAD_MUTEX_INITIALIZER` instead of `isc_mutex_init()` callWhen working on C11 thread library support, I found two cases where we use pthread specific semantics:
* Use of `void *` as a return type
* Use of `PTHREAD_MUTEX_INITIALIZER` instead of `isc_mutex_init()` callJanuary 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1552"geoip-use-ecs" broken with libmaxminddb 1.4.0+2020-01-13T14:07:09ZMichał Kępień"geoip-use-ecs" broken with libmaxminddb 1.4.0+An [upstream code change][1] introduced in libmaxminddb 1.4.0 causes the `geoip2` system test on *v9_11* to consistently fail due to the libmaxminddb library apparently thinking that 127.0.0.1 matches the `::10.53.0.1/128` IP range:
```...An [upstream code change][1] introduced in libmaxminddb 1.4.0 causes the `geoip2` system test on *v9_11* to consistently fail due to the libmaxminddb library apparently thinking that 127.0.0.1 matches the `::10.53.0.1/128` IP range:
```
$ grep . bin/tests/system/geoip2/dig.out.ns2.test31.*
bin/tests/system/geoip2/dig.out.ns2.test31.1:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.2:"2"
bin/tests/system/geoip2/dig.out.ns2.test31.3:"3"
bin/tests/system/geoip2/dig.out.ns2.test31.4:"4"
bin/tests/system/geoip2/dig.out.ns2.test31.5:"5"
bin/tests/system/geoip2/dig.out.ns2.test31.6:"6"
bin/tests/system/geoip2/dig.out.ns2.test31.7:"7"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.1:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.2:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.3:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.4:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.5:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.6:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.7:"1"
```
I know nothing about how libmaxminddb matches stuff internally, neither have I had the time to examine what the aforementioned upstream change does. This may even be an upstream bug in the library, but I would rather exercise caution before telling folks something they wrote does not work as intended when I am not 100% sure what those intentions are ;) To a novice like me, it looks like the library might now be treating an address from an unknown IP range as matching the first IP range defined in the database?
I tried rebuilding the `*.mmdb` files from `*.json` files in `bin/tests/system/geoip2/data/` using the `write-test-data.pl` script, but that did not change anything.
While this report only strictly applies to 9.11 because that is the only branch where the failing test still exists (`use-geoip-ecs` was ripped out in later releases), perhaps it is a symptom of an unexpected behavior which is *not* limited to our tests, but rather one that is occurring with newer versions of libmaxminddb in general (and we are simply unaware of it)?
@each, do you have any ideas on how to approach this?
[1]: https://github.com/maxmind/libmaxminddb/commit/2d49f4f04dad3103db701921258f3fac3b16ed80January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1547Out-of-tree build fails with uverr2result.c:15:10: fatal error: isc/platform....2020-01-09T09:24:41ZMichal NowakOut-of-tree build fails with uverr2result.c:15:10: fatal error: isc/platform.h: No such file or directoryOut-of-tree build fails with:
```
libtool: compile: gcc -include /builds/isc-projects/bind9/workspace/config.h -I/builds/isc-projects/bind9/workspace -I../../../.. -I../../../../lib/isc/netmgr/../include -I../../../../lib/isc/netmgr/../...Out-of-tree build fails with:
```
libtool: compile: gcc -include /builds/isc-projects/bind9/workspace/config.h -I/builds/isc-projects/bind9/workspace -I../../../.. -I../../../../lib/isc/netmgr/../include -I../../../../lib/isc/netmgr/../unix/include -I../../../../lib/isc/netmgr/../pthreads/include -I../../../../lib/isc/netmgr/.. -I/usr/include/json-c -I/usr/include/libxml2 -DISC_MEM_DEFAULTFILL=1 -DISC_LIST_CHECKINIT=1 -fno-omit-frame-pointer -fno-optimize-sibling-calls -O1 -g -Wall -Wextra -O3 -pthread -I/usr/include/google -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -Wno-missing-field-initializers -fno-strict-aliasing -Wshadow -Werror -c ../../../../lib/isc/netmgr/tcp.c -fPIC -DPIC -o .libs/tcp.o
../../../../lib/isc/netmgr/uverr2result.c:15:10: fatal error: isc/platform.h: No such file or directory
15 | #include <isc/platform.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [Makefile:185: uverr2result.lo] Error 1
In file included from ../../../../lib/isc/netmgr/../include/isc/buffer.h:105,
from ../../../../lib/isc/netmgr/udp.c:16:
../../../../lib/isc/netmgr/../include/isc/assertions.h:20:10: fatal error: isc/platform.h: No such file or directory
20 | #include <isc/platform.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [Makefile:185: udp.lo] Error 1
In file included from ../../../../lib/isc/netmgr/../include/isc/buffer.h:105,
from ../../../../lib/isc/netmgr/tcpdns.c:16:
../../../../lib/isc/netmgr/../include/isc/assertions.h:20:10: fatal error: isc/platform.h: No such file or directory
20 | #include <isc/platform.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
In file included from ../../../../lib/isc/netmgr/../include/isc/buffer.h:105,
from ../../../../lib/isc/netmgr/netmgr.c:17:
../../../../lib/isc/netmgr/../include/isc/assertions.h:20:10: fatal error: isc/platform.h: No such file or directory
20 | #include <isc/platform.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
In file included from ../../../../lib/isc/netmgr/../include/isc/buffer.h:105,
from ../../../../lib/isc/netmgr/tcp.c:17:
../../../../lib/isc/netmgr/../include/isc/assertions.h:20:10: fatal error: isc/platform.h: No such file or directory
20 | #include <isc/platform.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [Makefile:185: tcpdns.lo] Error 1
make[3]: *** [Makefile:185: tcp.lo] Error 1
make[3]: *** [Makefile:185: netmgr.lo] Error 1
make[3]: Target 'all' not remade because of errors.
make[3]: Leaving directory '/builds/isc-projects/bind9/workspace/lib/isc/netmgr'
make[2]: *** [Makefile:212: subdirs] Error 1
```
We had a similar one recently in 9.14 branch: https://gitlab.isc.org/isc-projects/bind9/issues/1530.January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Michal NowakMichal Nowakhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1546Add out-of-tree build to CI2020-01-09T09:43:43ZMichal NowakAdd out-of-tree build to CIIt seems that all our CI build jobs build BIND while in it's sources, hence https://gitlab.isc.org/isc-projects/bind9/issues/1530 was present for some time and only identified when the BIND package was being updated in distributions, whi...It seems that all our CI build jobs build BIND while in it's sources, hence https://gitlab.isc.org/isc-projects/bind9/issues/1530 was present for some time and only identified when the BIND package was being updated in distributions, which happened to build out-of-tree.
We should add an out-of-tree build job to the CI.January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Michal NowakMichal Nowakhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1530lib/dns/gen.c:29:26: fatal error: isc/platform.h: No such file or directory2020-01-07T11:37:53ZMichal Nowaklib/dns/gen.c:29:26: fatal error: isc/platform.h: No such file or directoryIn OpenIndiana (an illumos distribution) we build BIND out-of-tree and BIND 9.14.9 fails to build for me with:
```
making all in /userland/ws/oi-userland/components/network/bind/build/amd64/lib/dns
make[3]: Entering directory '/userland/...In OpenIndiana (an illumos distribution) we build BIND out-of-tree and BIND 9.14.9 fails to build for me with:
```
making all in /userland/ws/oi-userland/components/network/bind/build/amd64/lib/dns
make[3]: Entering directory '/userland/ws/oi-userland/components/network/bind/build/amd64/lib/dns'
/usr/gcc/6/bin/gcc -m64 -O3 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__=1 -D_XPG6 -D_POSIX_PTHREAD_SEMANTICS -pthread -I/usr/include/libxml2 -fPIC -I/userland/ws/oi-userland/components/network/bind/bind-9.14.9/lib/isc/include \
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 \
-m64 -Wl,-ztext -o gen /userland/ws/oi-userland/components/network/bind/bind-9.14.9/lib/dns/gen.c \
-lnsl -lsocket -lscf -ljson-c -llmdb -lxml2 -lz -llzma -lpthread -lm -lsocket -lnsl
/userland/ws/oi-userland/components/network/bind/bind-9.14.9/lib/dns/gen.c:29:26: fatal error: isc/platform.h: No such file or directory
#include <isc/platform.h>
^
```
The culprit seems to originated in d10fbdec84b5cbe2e51e0b74f41716dcc2c7035d.
I was able to make the problem away with the following patch:
```patch
--- bind-9.14.9/lib/dns/Makefile.in 2019-12-12 07:12:17.000000000 +0000
+++ bind-9.14.9/lib/dns/Makefile.in 2019-12-22 15:48:17.354544762 +0000
@@ -172,7 +172,7 @@ code.h: gen
./gen -s ${srcdir} > code.h || { rm -f $@ ; exit 1; }
gen: gen.c
- ${BUILD_CC} ${BUILD_CFLAGS} -I${top_srcdir}/lib/isc/include \
+ ${BUILD_CC} ${BUILD_CFLAGS} -I../../lib/isc/include \
${LFS_CFLAGS} ${LFS_LDFLAGS} \
${BUILD_CPPFLAGS} ${BUILD_LDFLAGS} -o $@ ${srcdir}/gen.c \
${BUILD_LIBS} ${LFS_LIBS}
```
If it's indeed an out-of-tree build issue, respective CI job might we worth.January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1525inline system test failed, need to wait for zone to be loaded.2020-01-14T13:56:57ZMark Andrewsinline system test failed, need to wait for zone to be loaded.Job [#493996](https://gitlab.isc.org/isc-projects/bind9/-/jobs/493996) failed for 41d827893e2b8529738491e4780fd76039ccf291:
```
I:inline:checking that records added from a journal are scheduled to be resigned (74)
I:inline:failed
I:inli...Job [#493996](https://gitlab.isc.org/isc-projects/bind9/-/jobs/493996) failed for 41d827893e2b8529738491e4780fd76039ccf291:
```
I:inline:checking that records added from a journal are scheduled to be resigned (74)
I:inline:failed
I:inline:check that zonestatus reports 'type: master' for a inline master zone (75)
I:inline:check that zonestatus reports 'type: slave' for a inline slave zone (76)
I:inline:exit status: 1
R:inline:FAIL
E:inline:Fri Dec 20 08:36:39 UTC 2019
```
```
[beetle:bin/tests/system] marka% !mo
more inline/rndc.out.ns3.post.test74
lt-rndc: 'zonestatus' failed: zone not loaded
[beetle:bin/tests/system] marka%
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1524stats.c:300:29:error: undeclared (first use in this function): stats->counter...2019-12-19T22:41:14ZMichal Nowakstats.c:300:29:error: undeclared (first use in this function): stats->counters[counter] = val;Build of BIND 9.11.14 on ppc64le CentOS 7 Copr build root fails with:
```
/bin/sh /builddir/build/BUILD/bind-9.11.14/libtool --mode=compile --tag=CC gcc -std=gnu99 -I/builddir/build/BUILD/bind-9.11.14 -I../.. -I./unix/include -I./pthrea...Build of BIND 9.11.14 on ppc64le CentOS 7 Copr build root fails with:
```
/bin/sh /builddir/build/BUILD/bind-9.11.14/libtool --mode=compile --tag=CC gcc -std=gnu99 -I/builddir/build/BUILD/bind-9.11.14 -I../.. -I./unix/include -I./pthreads/include -I./noatomic/include -I./include -I./include -I/builddir/build/BUILD/bind-9.11.14/lib/dns/include -I../../lib/dns/include -I/opt/isc/isc-bind/root/usr/include -D_REENTRANT -DOPENSSL -DPK11_LIB_LOCATION=\"undefined\" -D_GNU_SOURCE -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -I/usr/include/libxml2 -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -fno-strict-aliasing -fno-delete-null-pointer-checks -c stats.c
libtool: compile: gcc -std=gnu99 -I/builddir/build/BUILD/bind-9.11.14 -I../.. -I./unix/include -I./pthreads/include -I./noatomic/include -I./include -I./include -I/builddir/build/BUILD/bind-9.11.14/lib/dns/include -I../../lib/dns/include -I/opt/isc/isc-bind/root/usr/include -D_REENTRANT -DOPENSSL -DPK11_LIB_LOCATION=\"undefined\" -D_GNU_SOURCE -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -I/usr/include/libxml2 -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -fno-strict-aliasing -fno-delete-null-pointer-checks -c stats.c -fPIC -DPIC -o .libs/stats.o
�[01m�[Kstats.c:�[m�[K In function '�[01m�[Ksetcounter�[m�[K':
�[01m�[Kstats.c:300:29:�[m�[K �[01;31m�[Kerror: �[m�[K'�[01m�[Kval�[m�[K' undeclared (first use in this function)
stats->counters[counter] = val;
�[01;32m�[K ^�[m�[K
�[01m�[Kstats.c:300:29:�[m�[K �[01;36m�[Knote: �[m�[Keach undeclared identifier is reported only once for each function it appears in
�[01m�[Kstats.c:286:20:�[m�[K �[01;35m�[Kwarning: �[m�[Kunused parameter '�[01m�[Kvalue�[m�[K' [-Wunused-parameter]
const uint64_t value)
�[01;32m�[K ^�[m�[K
make[2]: *** [stats.lo] Error 1
make[2]: Leaving directory `/builddir/build/BUILD/bind-9.11.14/lib/isc'
make[1]: *** [subdirs] Error 1
make[1]: Leaving directory `/builddir/build/BUILD/bind-9.11.14/lib'
make: *** [subdirs] Error 1
error: Bad exit status from /var/tmp/rpm-tmp.m7mn1J (%build)
```
See https://copr-be.cloud.fedoraproject.org/results/isc/bind-esv/srpm-builds/01131112/builder-live.log.gz (also attached as [builder-live.log](/uploads/04bea65eacd81485c149868a0b630355/builder-live.log)).
Recently changed in e50abe8a70e by @ondrej.January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1523pkcs11-destroy's usage message is misleading2019-12-20T09:06:34ZMichal Nowakpkcs11-destroy's usage message is misleading`pkcs11-destroy`'s usage message says "use whatever options you want, if any":
```
# /root/bind9/bin/pkcs11/pkcs11-destroy -h
Unrecognised option: -h
Usage:
pkcs11-destroy [-m module] [-s slot] [-i id | -l label] [-p pin] [-w waittime]
...`pkcs11-destroy`'s usage message says "use whatever options you want, if any":
```
# /root/bind9/bin/pkcs11/pkcs11-destroy -h
Unrecognised option: -h
Usage:
pkcs11-destroy [-m module] [-s slot] [-i id | -l label] [-p pin] [-w waittime]
```
But `pkcs11-destroy(8)` man page says that either `-i`, or `-l` needs to be provided:
```
SYNOPSIS
pkcs11-destroy [-m module] [-s slot] {-i ID | -l label} [-p PIN] [-w seconds]
DESCRIPTION
pkcs11-destroy destroys keys stored in a PKCS#11 device, identified by their ID or label.
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1513inline system test failed2020-01-08T07:32:00ZMark Andrewsinline system test failedJob [#485428](https://gitlab.isc.org/isc-projects/bind9/-/jobs/485428) failed for 68b9df92ac9217e06b0e30fb9f4e946385906040:
```
I:inline:testing updating dynamic serial via 'rndc signing -serial' (64)
I:inline:failed
I:inline:testing up...Job [#485428](https://gitlab.isc.org/isc-projects/bind9/-/jobs/485428) failed for 68b9df92ac9217e06b0e30fb9f4e946385906040:
```
I:inline:testing updating dynamic serial via 'rndc signing -serial' (64)
I:inline:failed
I:inline:testing updating dynamic serial via 'rndc signing -serial' with negative change (65)
```
```
[beetle:~/git/bind9/art] marka% cat bin/tests/system/inline/dig.out.ns2.pre.test64
; <<>> DiG 9.15.7 <<>> +tcp +dnssec -p 8600 bits. SOA @10.53.0.2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6398
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; COOKIE: d002aaae9aa2d2bb010000005df81fd0cd03a161bb275167 (good)
;; QUESTION SECTION:
;bits. IN SOA
;; ANSWER SECTION:
bits. 0 IN SOA ns2.bits. . 2011072460 20 20 1814400 3600
;; Query time: 8 msec
;; SERVER: 10.53.0.2#8600(10.53.0.2)
;; WHEN: Tue Dec 17 00:22:40 UTC 2019
;; MSG SIZE rcvd: 100
[beetle:~/git/bind9/art] marka%
```
```
[beetle:~/git/bind9/art] marka% cat bin/tests/system/inline/dig.out.ns2.post.test64
; <<>> DiG 9.15.7 <<>> +tcp +dnssec -p 8600 bits. SOA @10.53.0.2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12194
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; COOKIE: bcfc93baa244520a010000005df81fd1404c83bd457e385e (good)
;; QUESTION SECTION:
;bits. IN SOA
;; ANSWER SECTION:
bits. 0 IN SOA ns2.bits. . 2011072460 20 20 1814400 3600
;; Query time: 8 msec
;; SERVER: 10.53.0.2#8600(10.53.0.2)
;; WHEN: Tue Dec 17 00:22:41 UTC 2019
;; MSG SIZE rcvd: 100
[beetle:~/git/bind9/art] marka%
```
```
17-Dec-2019 00:22:40.182 received control channel command 'signing -serial 2011072470 bits'
17-Dec-2019 00:22:40.182 socket 0x7f358802d008: socket_recv: event 0x7f35c5273440 -> task 0x7f35c5272020
17-Dec-2019 00:22:40.182 setserial: zone bits/IN: enter
17-Dec-2019 00:22:40.182 zone bits/IN: could not get zone keys for secure dynamic update
17-Dec-2019 00:22:40.182 zone_journal: zone bits/IN: enter
17-Dec-2019 00:22:40.182 writing to journal
17-Dec-2019 00:22:40.182 del bits. 0 IN SOA ns2.bits. . 2011072460 20 20 1814400 3600
17-Dec-2019 00:22:40.182 add bits. 0 IN SOA ns2.bits. . 2011072470 20 20 1814400 3600
17-Dec-2019 00:22:40.182 sockmgr 0x7f35c5260020 thread 0: watcher got message -3 for socket 64
17-Dec-2019 00:22:40.182 sockmgr 0x7f35c5260020 thread 0: watcher got message -2 for socket -1
17-Dec-2019 00:22:40.182 socket 0x7f358802d008: internal_recv: event 0x7f35c5273440 -> task 0x7f35c5272020
17-Dec-2019 00:22:41.838 clientmgr @0x7f35c51ed020: clientmctx
17-Dec-2019 00:22:41.838 clientmgr @0x7f35c51ed020 attach: 18
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (<unknown-query>): query_reset
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 (no-peer): ns_client_newnamebuf
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 (no-peer): ns_client_newnamebuf: done
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 (no-peer): client_setup
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 (no-peer): allocate new client
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369: TCP request
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369: using view '_default'
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369: request is not signed
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369: recursion not available
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369: query
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (<unknown-query>): ns_query_start
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): qctx_init
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): client attr:0x22311, query attr:0x700, restarts:0, origqname:bits, timer:0, authdb:0, referral:0
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): ns__query_start
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): query 'bits/SOA/IN' approved
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_lookup
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): ns_client_getnamebuf
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): ns_client_getnamebuf: done
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): ns_client_newname
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): ns_client_newname: done
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_gotanswer
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): rrl=(nil), HAVECOOKIE=0, result=ISC_R_SUCCESS, fname=0x7f358c01a2e0(1), is_zone=1, RECURSIONOK=0, query.rpz_st=(nil)(0), RRL_CHECKED=0
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_checkrpz
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): rpz_rewrite
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_prepresponse
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_zerottl_refetch
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_respond
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_getexpire
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_addanswer
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_addrrset
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): ns_client_keepname
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_setorder
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_additional
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_additional: done
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_addrrset: done
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_addnoqnameproof
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_addauth
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): ns_query_done
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): send
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): senddone
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): reset client
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): endrequest
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_reset
17-Dec-2019 00:22:41.846 client @0x7f358c057bb0 10.53.0.1#53369: freeing client
17-Dec-2019 00:22:42.038 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (<unknown-query>): query_reset
17-Dec-2019 00:22:42.038 clientmgr @0x7f35c51ed020 detach: 17
17-Dec-2019 00:22:42.046 zone_needdump: zone bits/IN: enter
17-Dec-2019 00:22:42.046 zone_settimer: zone bits/IN: enter
17-Dec-2019 00:22:42.078 socket 0x7f358802d008: destroying
17-Dec-2019 00:22:42.078 sockmgr 0x7f35c5260020 thread 0: watcher got message -5 for socket 64
17-Dec-2019 00:22:42.078 sockmgr 0x7f35c5260020 thread 0: watcher got message -2 for socket -1
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/1507ThreadSanitizer: data race - lib/dns/tests/zt_test2020-01-13T11:11:14ZMark AndrewsThreadSanitizer: data race - lib/dns/tests/zt_testlib/dns/tests/zt_test failed
```
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by main thread (mutexes: write M1, write M2):
#0 free <null>
#1 default_memfree lib/isc/mem.c:713:2
#2 mem_put lib/isc...lib/dns/tests/zt_test failed
```
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by main thread (mutexes: write M1, write M2):
#0 free <null>
#1 default_memfree lib/isc/mem.c:713:2
#2 mem_put lib/isc/mem.c:627:2
#3 isc___mem_putanddetach lib/isc/mem.c:960:3
#4 isc__mem_putanddetach lib/isc/mem.c:2380:2
#5 zt_destroy lib/dns/zt.c:233:2
#6 zt_flushanddetach lib/dns/zt.c:250:3
#7 dns_zt_detach lib/dns/zt.c:261:2
#8 view_flushanddetach lib/dns/view.c:594:5
#9 dns_view_detach lib/dns/view.c:635:2
#10 asyncload_zt lib/dns/tests/zt_test.c:339:2
#11 <null> <null>
#12 __libc_start_main <null>
Previous atomic write of size 8 at 0x000000000001 by thread T1:
#0 __tsan_atomic64_fetch_sub <null>
#1 doneloading lib/dns/zt.c:551:6
#2 zone_asyncload lib/dns/zone.c:2201:3
#3 dispatch lib/isc/task.c:1134:7
#4 run lib/isc/task.c:1319:2
Mutex M1 (0x000000000020) created at:
#0 pthread_mutex_init <null>
#1 isc__mutex_init lib/isc/pthreads/mutex.c:284:8
#2 dns_view_create lib/dns/view.c:107:2
#3 dns_test_makeview lib/dns/tests/dnstest.c:225:2
#4 dns_test_makezone lib/dns/tests/dnstest.c:273:12
#5 asyncload_zt lib/dns/tests/zt_test.c:275:11
#6 <null> <null>
#7 __libc_start_main <null>
Mutex M2 (0x000000000027) created at:
#0 pthread_mutex_init <null>
#1 isc__mutex_init lib/isc/pthreads/mutex.c:284:8
#2 mem_create lib/isc/mem.c:737:2
#3 isc_mem_create lib/isc/mem.c:2358:2
#4 dns_test_begin lib/dns/tests/dnstest.c:145:2
#5 _setup lib/dns/tests/zt_test.c:55:11
#6 <null> <null>
#7 __libc_start_main <null>
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:75:8
#2 isc_taskmgr_create lib/isc/task.c:1410:3
#3 create_managers lib/dns/tests/dnstest.c:119:2
#4 dns_test_begin lib/dns/tests/dnstest.c:180:3
#5 _setup lib/dns/tests/zt_test.c:55:11
#6 <null> <null>
#7 __libc_start_main <null>
SUMMARY: ThreadSanitizer: data race in free
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/1500Be more verbose when opening IPC socket or fsyncing dir fails2020-01-13T21:50:40ZWitold KrecickiBe more verbose when opening IPC socket or fsyncing dir failsJanuary 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Witold KrecickiWitold Krecickihttps://gitlab.isc.org/isc-projects/bind9/-/issues/1499Deadlock when TCP listening fails2020-01-13T21:50:38ZWitold KrecickiDeadlock when TCP listening failsJanuary 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Witold KrecickiWitold Krecickihttps://gitlab.isc.org/isc-projects/bind9/-/issues/1497ThreadSanitizer: data race lib/isc/unix/socket.c:1839:2 in destroy2019-12-12T14:14:48ZOndřej SurýThreadSanitizer: data race lib/isc/unix/socket.c:1839:2 in destroy```
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1:
#0 destroy lib/isc/unix/socket.c:1839:2
#1 process_fd lib/isc/unix/socket.c:3177:3
#2 process_fds lib/isc/unix/socket.c:3257:3
#3 ne...```
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1:
#0 destroy lib/isc/unix/socket.c:1839:2
#1 process_fd lib/isc/unix/socket.c:3177:3
#2 process_fds lib/isc/unix/socket.c:3257:3
#3 netthread lib/isc/unix/socket.c:3503:10
Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M1, write M2):
#0 send_recvdone_event lib/isc/unix/socket.c
#1 isc_socket_cancel lib/isc/unix/socket.c:5053:5
#2 dns_dispatch_detach lib/dns/dispatch.c:2985:4
#3 fctx_cancelquery lib/dns/resolver.c:1378:3
#4 rctx_done lib/dns/resolver.c:9478:3
#5 resquery_response lib/dns/resolver.c
#6 dispatch lib/isc/task.c:1136:7
#7 run lib/isc/task.c:1323:2
Location is heap block of size 361 at 0x000000000014 allocated by thread T2:
#0 malloc <null>
#1 default_memalloc lib/isc/mem.c:685:8
#2 mem_get lib/isc/mem.c:598:8
#3 isc___mem_get lib/isc/mem.c:1013:9
#4 isc__mem_get lib/isc/mem.c:2365:10
#5 allocate_socket lib/isc/unix/socket.c:1876:9
#6 socket_create lib/isc/unix/socket.c:2450:11
#7 isc_socket_create lib/isc/unix/socket.c:2535:10
#8 fctx_query lib/dns/resolver.c:2044:12
#9 rctx_resend lib/dns/resolver.c:9376:11
#10 rctx_done lib/dns/resolver.c:9497:3
#11 resquery_response lib/dns/resolver.c
#12 dispatch lib/isc/task.c:1136:7
#13 run lib/isc/task.c:1323:2
Mutex M1 is already destroyed.
Mutex M2 (0x000000000026) created at:
#0 pthread_mutex_init <null>
#1 isc__mutex_init lib/isc/pthreads/mutex.c:284:8
#2 allocate_socket lib/isc/unix/socket.c:1913:2
#3 socket_create lib/isc/unix/socket.c:2450:11
#4 isc_socket_create lib/isc/unix/socket.c:2535:10
#5 fctx_query lib/dns/resolver.c:2044:12
#6 rctx_resend lib/dns/resolver.c:9376:11
#7 rctx_done lib/dns/resolver.c:9497:3
#8 resquery_response lib/dns/resolver.c
#9 dispatch lib/isc/task.c:1136:7
#10 run lib/isc/task.c:1323:2
Thread T2 'isc-socket-0' (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:75:8
#2 isc_socketmgr_create2 lib/isc/unix/socket.c:3817:3
#3 isc_socketmgr_create lib/isc/unix/socket.c:3774:10
#4 isc_socketmgr_createinctx lib/isc/unix/socket.c:5561:11
#5 main bin/delv/delv.c:1731:2
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:75:8
#2 isc_taskmgr_create lib/isc/task.c:1414:3
#3 isc_taskmgr_createinctx lib/isc/task.c:1982:11
#4 main bin/delv/delv.c:1730:2
SUMMARY: ThreadSanitizer: data race lib/isc/unix/socket.c:1839:2 in destroy
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1492Backports missing from December maintenance releases2020-01-10T10:22:03ZMichał KępieńBackports missing from December maintenance releasesHere is a list[^1] of merge requests (along with their branch names) which, according to my brief examination, should be backported from *master* to *v9_14* and *v9_11*, but have not yet been backported. Almost all of them only fix test...Here is a list[^1] of merge requests (along with their branch names) which, according to my brief examination, should be backported from *master* to *v9_14* and *v9_11*, but have not yet been backported. Almost all of them only fix tests and thus are not critical, but there is one outlier (**in bold**) about which I am not sure:
- [x] !2750 `michal/fix-the-forward-system-test-on-windows`
- [x] !2747 `1479-_wait_for_rcode-adds-extraneous-query`
- [x] !2736 `ondrej/run-full-pipeline-on-schedule`
- [x] !2712 `1453-the-zero-system-test-timeouts-intermittently`
- [x] !2716 `1458-intermittent-failure-in-the-forward-system-test`
- [x] !2714 `1425-intermittent-failure-in-the-addzone-system-test`
- [x] !2715 `1401-intermittent-failures-in-the-catz-system-test`
- [x] !2705 `1427-intermittent-failure-in-fetchlimit-system-test`
- [x] !2704 `1407-intermittent-failure-in-the-mkeys-system-test`
- [x] !2671 `1410-intermittent-failure-in-the-resolver-test`
- [x] !2669 `1431-summary-sanitizer-grep-is-dangerous`
- [x] !2657 `ondrej/keep-sanitizer-tainted-system-tests-files`
- [x] !2649 `ondrej/stop-retrying-system-tests`
- [x] !2633 `1401-intermittent-failures-in-the-catz-system-test`
- [x] !2636 `1402-multiple-issues-in-the-runtime-system-test`
- [x] !2634 `ondrej/detect-cores-in-system-tests-on-FreeBSD`
- [x] !2601 `1380-autosign-jitter-test-fails-with-no-nsec3param-found-in-axfr`
- [x] !2554 `1256-fix-the-jitter-test-2`
- [x] !2553 `1256-fix-the-jitter-test`
- [x] !2611 **`1394-incoming-zone-transfer-messages-logged-to-wrong-category`**
- [x] !2602 `ondrej/get-the-backtraces-out-of-system-test-coredumps`
The MR **in bold** is both a bug fix and a functional change, so we should clarify whether it should be backported or not.
I made a list of checkboxes for all the other ones so that we can conveniently track the outstanding ones as I would like all test-related MRs (or at least those which apply relatively cleanly) to be backported for the January releases.
[^1]: in reverse merge order, i.e. starting from the most recently merged oneJanuary 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Michał KępieńMichał Kępieńhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1486ThreadSanitizer: lock-order-inversion (potential deadlock) - dns_resolver_cre...2019-12-13T20:52:32ZOndřej SurýThreadSanitizer: lock-order-inversion (potential deadlock) - dns_resolver_createfetch vs dns_resolver_shutdown```
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=25300)
Cycle in lock order graph: M1131 (0x7b7400000238) => M1728 (0x7b4c000001d0) => M1131
Mutex M1728 acquired here while holding mutex M1131 in thread T...```
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=25300)
Cycle in lock order graph: M1131 (0x7b7400000238) => M1728 (0x7b4c000001d0) => M1131
Mutex M1728 acquired here while holding mutex M1131 in thread T1:
#0 pthread_mutex_lock <null> (delv+0x4492a6)
#1 fcount_incr /home/ondrej/Projects/bind9/lib/dns/resolver.c:1524:2 (libdns.so.1505+0x15b6ab)
#2 fctx_create /home/ondrej/Projects/bind9/lib/dns/resolver.c:4928:11 (libdns.so.1505+0x156959)
#3 dns_resolver_createfetch /home/ondrej/Projects/bind9/lib/dns/resolver.c:10588:12 (libdns.so.1505+0x15435e)
#4 create_fetch /home/ondrej/Projects/bind9/lib/dns/validator.c:1055:10 (libdns.so.1505+0x1a239a)
#5 seek_dnskey /home/ondrej/Projects/bind9/lib/dns/validator.c:1305:12 (libdns.so.1505+0x1a2f84)
#6 validate_answer /home/ondrej/Projects/bind9/lib/dns/validator.c:1557:13 (libdns.so.1505+0x1a06b3)
#7 validator_start /home/ondrej/Projects/bind9/lib/dns/validator.c:3157:13 (libdns.so.1505+0x19f013)
#8 dispatch /home/ondrej/Projects/bind9/lib/isc/task.c:1134:7 (libisc.so.1504+0x52007)
#9 run /home/ondrej/Projects/bind9/lib/isc/task.c:1319:2 (libisc.so.1504+0x4f3a8)
Mutex M1131 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (delv+0x4492a6)
#1 dns_resolver_createfetch /home/ondrej/Projects/bind9/lib/dns/resolver.c:10545:2 (libdns.so.1505+0x1540f4)
#2 create_fetch /home/ondrej/Projects/bind9/lib/dns/validator.c:1055:10 (libdns.so.1505+0x1a239a)
#3 seek_dnskey /home/ondrej/Projects/bind9/lib/dns/validator.c:1305:12 (libdns.so.1505+0x1a2f84)
#4 validate_answer /home/ondrej/Projects/bind9/lib/dns/validator.c:1557:13 (libdns.so.1505+0x1a06b3)
#5 validator_start /home/ondrej/Projects/bind9/lib/dns/validator.c:3157:13 (libdns.so.1505+0x19f013)
#6 dispatch /home/ondrej/Projects/bind9/lib/isc/task.c:1134:7 (libisc.so.1504+0x52007)
#7 run /home/ondrej/Projects/bind9/lib/isc/task.c:1319:2 (libisc.so.1504+0x4f3a8)
Mutex M1131 acquired here while holding mutex M1728 in main thread:
#0 pthread_mutex_lock <null> (delv+0x4492a6)
#1 dns_resolver_shutdown /home/ondrej/Projects/bind9/lib/dns/resolver.c:10312:4 (libdns.so.1505+0x155107)
#2 view_flushanddetach /home/ondrej/Projects/bind9/lib/dns/view.c:582:4 (libdns.so.1505+0x1a7031)
#3 dns_view_detach /home/ondrej/Projects/bind9/lib/dns/view.c:635:2 (libdns.so.1505+0x1a72eb)
#4 destroyclient /home/ondrej/Projects/bind9/lib/dns/client.c:611:3 (libdns.so.1505+0x213882)
#5 dns_client_destroy /home/ondrej/Projects/bind9/lib/dns/client.c:652:3 (libdns.so.1505+0x21369b)
#6 main /home/ondrej/Projects/bind9/bin/delv/delv.c:1827:3 (delv+0x4b9926)
Mutex M1728 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (delv+0x4492a6)
#1 dns_resolver_shutdown /home/ondrej/Projects/bind9/lib/dns/resolver.c:10307:2 (libdns.so.1505+0x155004)
#2 view_flushanddetach /home/ondrej/Projects/bind9/lib/dns/view.c:582:4 (libdns.so.1505+0x1a7031)
#3 dns_view_detach /home/ondrej/Projects/bind9/lib/dns/view.c:635:2 (libdns.so.1505+0x1a72eb)
#4 destroyclient /home/ondrej/Projects/bind9/lib/dns/client.c:611:3 (libdns.so.1505+0x213882)
#5 dns_client_destroy /home/ondrej/Projects/bind9/lib/dns/client.c:652:3 (libdns.so.1505+0x21369b)
#6 main /home/ondrej/Projects/bind9/bin/delv/delv.c:1827:3 (delv+0x4b9926)
Thread T1 'isc-worker0000' (tid=25315, running) created by main thread at:
#0 pthread_create <null> (delv+0x42b04b)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/pthreads/thread.c:75:8 (libisc.so.1504+0x6a9ea)
#2 isc_taskmgr_create /home/ondrej/Projects/bind9/lib/isc/task.c:1410:3 (libisc.so.1504+0x4f1b2)
#3 isc_taskmgr_createinctx /home/ondrej/Projects/bind9/lib/isc/task.c:1978:11 (libisc.so.1504+0x51a92)
#4 main /home/ondrej/Projects/bind9/bin/delv/delv.c:1730:2 (delv+0x4b93d9)
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/home/ondrej/Projects/bind9/bin/delv/.libs/delv+0x4492a6) in pthread_mutex_lock
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1482autosign system test failed2019-12-13T08:58:19ZMark Andrewsautosign system test failedJob [#467197](https://gitlab.isc.org/isc-projects/bind9/-/jobs/467197) failed for 1afc984ba892ba9ae19051f00203b105299d21ed:Job [#467197](https://gitlab.isc.org/isc-projects/bind9/-/jobs/467197) failed for 1afc984ba892ba9ae19051f00203b105299d21ed:January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1476ThreadSanitizer: data race lib/isc/log.c - multiple issues2020-03-10T12:11:24ZOndřej SurýThreadSanitizer: data race lib/isc/log.c - multiple issuesFound in `logfileconfig` test:
```
WARNING: ThreadSanitizer: data race (pid=1260)
Read of size 8 at 0x7b9400000068 by thread T21 (mutexes: write M944203102091890528, write M246707628766581536):
#0 isc_log_wouldlog /home/ondrej/Pro...Found in `logfileconfig` test:
```
WARNING: ThreadSanitizer: data race (pid=1260)
Read of size 8 at 0x7b9400000068 by thread T21 (mutexes: write M944203102091890528, write M246707628766581536):
#0 isc_log_wouldlog /home/ondrej/Projects/bind9/lib/isc/log.c:1479 (libisc.so.1504+0x31379)
#1 socket_log /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:640 (libisc.so.1504+0x6c3b3)
#2 internal_accept /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:2784 (libisc.so.1504+0x70a08)
#3 process_fd /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3160 (libisc.so.1504+0x70a08)
#4 process_fds /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3257 (libisc.so.1504+0x70a08)
#5 netthread /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3503 (libisc.so.1504+0x70a08)
#6 <null> <null> (libtsan.so.0+0x29b3d)
Previous write of size 8 at 0x7b9400000068 by thread T9 (mutexes: write M29):
#0 isc_logconfig_use /home/ondrej/Projects/bind9/lib/isc/log.c:426 (libisc.so.1504+0x2f4d7)
#1 load_configuration server.c:9108 (named+0x5945a)
#2 run_server server.c:9654 (named+0x59a47)
#3 dispatch /home/ondrej/Projects/bind9/lib/isc/task.c:1134 (libisc.so.1504+0x56fa6)
#4 run /home/ondrej/Projects/bind9/lib/isc/task.c:1319 (libisc.so.1504+0x56fa6)
#5 <null> <null> (libtsan.so.0+0x29b3d)
Location is heap block of size 8321 at 0x7b9400000000 allocated by main thread:
#0 malloc <null> (libtsan.so.0+0x2b1a3)
#1 default_memalloc /home/ondrej/Projects/bind9/lib/isc/mem.c:685 (libisc.so.1504+0x33fee)
#2 mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:598 (libisc.so.1504+0x34c7e)
#3 mem_allocateunlocked /home/ondrej/Projects/bind9/lib/isc/mem.c:1222 (libisc.so.1504+0x34c7e)
#4 isc___mem_allocate /home/ondrej/Projects/bind9/lib/isc/mem.c:1242 (libisc.so.1504+0x34c7e)
#5 isc__mem_allocate /home/ondrej/Projects/bind9/lib/isc/mem.c:2387 (libisc.so.1504+0x3be64)
#6 isc___mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:1007 (libisc.so.1504+0x3c6ca)
#7 isc__mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:2365 (libisc.so.1504+0x3aef1)
#8 isc_log_create /home/ondrej/Projects/bind9/lib/isc/log.c:262 (libisc.so.1504+0x30731)
#9 named_log_init /home/ondrej/Projects/bind9/bin/named/log.c:61 (named+0x26f38)
#10 setup main.c:1043 (named+0x1a6db)
#11 main main.c:1515 (named+0x1a6db)
Mutex M944203102091890528 is already destroyed.
Mutex M246707628766581536 is already destroyed.
Mutex M29 (0x7b9400000040) created at:
#0 pthread_mutex_init <null> (libtsan.so.0+0x2c5ad)
#1 isc__mutex_init /home/ondrej/Projects/bind9/lib/isc/pthreads/mutex.c:284 (libisc.so.1504+0x7bc06)
#2 isc_log_create /home/ondrej/Projects/bind9/lib/isc/log.c:274 (libisc.so.1504+0x3081b)
#3 named_log_init /home/ondrej/Projects/bind9/bin/named/log.c:61 (named+0x26f38)
#4 setup main.c:1043 (named+0x1a6db)
#5 main main.c:1515 (named+0x1a6db)
Thread T21 'isc-socket-3' (tid=1350, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2be1b)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/pthreads/thread.c:75 (libisc.so.1504+0x7bcc4)
#2 isc_socketmgr_create2 /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3817 (libisc.so.1504+0x74ce2)
#3 create_managers main.c:919 (named+0x1af64)
#4 setup main.c:1235 (named+0x1af64)
#5 main main.c:1515 (named+0x1af64)
Thread T9 'isc-worker0000' (tid=1327, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2be1b)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/pthreads/thread.c:75 (libisc.so.1504+0x7bcc4)
#2 isc_taskmgr_create /home/ondrej/Projects/bind9/lib/isc/task.c:1410 (libisc.so.1504+0x59d63)
#3 create_managers main.c:902 (named+0x1aeec)
#4 setup main.c:1235 (named+0x1aeec)
#5 main main.c:1515 (named+0x1aeec)
SUMMARY: ThreadSanitizer: data race /home/ondrej/Projects/bind9/lib/isc/log.c:1479 in isc_log_wouldlog
```
```
WARNING: ThreadSanitizer: data race (pid=1260)
Read of size 4 at 0x7b18000059e0 by thread T21 (mutexes: write M944203102091890528, write M246707628766581536):
#0 isc_log_wouldlog /home/ondrej/Projects/bind9/lib/isc/log.c:1482 (libisc.so.1504+0x3138c)
#1 socket_log /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:640 (libisc.so.1504+0x6c3b3)
#2 internal_accept /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:2784 (libisc.so.1504+0x70a08)
#3 process_fd /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3160 (libisc.so.1504+0x70a08)
#4 process_fds /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3257 (libisc.so.1504+0x70a08)
#5 netthread /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3503 (libisc.so.1504+0x70a08)
#6 <null> <null> (libtsan.so.0+0x29b3d)
Previous write of size 4 at 0x7b18000059e0 by thread T9:
#0 assignchannel /home/ondrej/Projects/bind9/lib/isc/log.c:1000 (libisc.so.1504+0x2e13a)
#1 isc_log_usechannel /home/ondrej/Projects/bind9/lib/isc/log.c:786 (libisc.so.1504+0x30ab2)
#2 category_fromconf /home/ondrej/Projects/bind9/bin/named/logconf.c:75 (named+0x27872)
#3 named_logconfig /home/ondrej/Projects/bind9/bin/named/logconf.c:329 (named+0x27872)
#4 load_configuration server.c:9097 (named+0x5941b)
#5 run_server server.c:9654 (named+0x59a47)
#6 dispatch /home/ondrej/Projects/bind9/lib/isc/task.c:1134 (libisc.so.1504+0x56fa6)
#7 run /home/ondrej/Projects/bind9/lib/isc/task.c:1319 (libisc.so.1504+0x56fa6)
#8 <null> <null> (libtsan.so.0+0x29b3d)
Location is heap block of size 89 at 0x7b18000059a0 allocated by thread T9:
#0 malloc <null> (libtsan.so.0+0x2b1a3)
#1 default_memalloc /home/ondrej/Projects/bind9/lib/isc/mem.c:685 (libisc.so.1504+0x33fee)
#2 mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:598 (libisc.so.1504+0x34c7e)
#3 mem_allocateunlocked /home/ondrej/Projects/bind9/lib/isc/mem.c:1222 (libisc.so.1504+0x34c7e)
#4 isc___mem_allocate /home/ondrej/Projects/bind9/lib/isc/mem.c:1242 (libisc.so.1504+0x34c7e)
#5 isc__mem_allocate /home/ondrej/Projects/bind9/lib/isc/mem.c:2387 (libisc.so.1504+0x3be64)
#6 isc___mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:1007 (libisc.so.1504+0x3c6ca)
#7 isc__mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:2365 (libisc.so.1504+0x3aef1)
#8 isc_logconfig_create /home/ondrej/Projects/bind9/lib/isc/log.c:322 (libisc.so.1504+0x3035f)
#9 isc_logconfig_create /home/ondrej/Projects/bind9/lib/isc/log.c:313 (libisc.so.1504+0x3035f)
#10 load_configuration server.c:9091 (named+0x593a2)
#11 run_server server.c:9654 (named+0x59a47)
#12 dispatch /home/ondrej/Projects/bind9/lib/isc/task.c:1134 (libisc.so.1504+0x56fa6)
#13 run /home/ondrej/Projects/bind9/lib/isc/task.c:1319 (libisc.so.1504+0x56fa6)
#14 <null> <null> (libtsan.so.0+0x29b3d)
Mutex M944203102091890528 is already destroyed.
Mutex M246707628766581536 is already destroyed.
Thread T21 'isc-socket-3' (tid=1350, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2be1b)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/pthreads/thread.c:75 (libisc.so.1504+0x7bcc4)
#2 isc_socketmgr_create2 /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3817 (libisc.so.1504+0x74ce2)
#3 create_managers main.c:919 (named+0x1af64)
#4 setup main.c:1235 (named+0x1af64)
#5 main main.c:1515 (named+0x1af64)
Thread T9 'isc-worker0000' (tid=1327, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2be1b)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/pthreads/thread.c:75 (libisc.so.1504+0x7bcc4)
#2 isc_taskmgr_create /home/ondrej/Projects/bind9/lib/isc/task.c:1410 (libisc.so.1504+0x59d63)
#3 create_managers main.c:902 (named+0x1aeec)
#4 setup main.c:1235 (named+0x1aeec)
#5 main main.c:1515 (named+0x1aeec)
SUMMARY: ThreadSanitizer: data race /home/ondrej/Projects/bind9/lib/isc/log.c:1482 in isc_log_wouldlog
```
```
WARNING: ThreadSanitizer: data race (pid=1260)
Read of size 1 at 0x7b18000059f0 by thread T21 (mutexes: write M944203102091890528, write M246707628766581536):
#0 isc_log_wouldlog /home/ondrej/Projects/bind9/lib/isc/log.c:1483 (libisc.so.1504+0x313b9)
#1 socket_log /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:640 (libisc.so.1504+0x6c3b3)
#2 internal_accept /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:2784 (libisc.so.1504+0x70a08)
#3 process_fd /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3160 (libisc.so.1504+0x70a08)
#4 process_fds /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3257 (libisc.so.1504+0x70a08)
#5 netthread /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3503 (libisc.so.1504+0x70a08)
#6 <null> <null> (libtsan.so.0+0x29b3d)
Previous write of size 8 at 0x7b18000059f0 by thread T9 (mutexes: write M15):
#0 malloc <null> (libtsan.so.0+0x2b1a3)
#1 default_memalloc /home/ondrej/Projects/bind9/lib/isc/mem.c:685 (libisc.so.1504+0x33fee)
#2 mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:598 (libisc.so.1504+0x34c7e)
#3 mem_allocateunlocked /home/ondrej/Projects/bind9/lib/isc/mem.c:1222 (libisc.so.1504+0x34c7e)
#4 isc___mem_allocate /home/ondrej/Projects/bind9/lib/isc/mem.c:1242 (libisc.so.1504+0x34c7e)
#5 isc__mem_allocate /home/ondrej/Projects/bind9/lib/isc/mem.c:2387 (libisc.so.1504+0x3be64)
#6 isc___mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:1007 (libisc.so.1504+0x3c6ca)
#7 isc__mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:2365 (libisc.so.1504+0x3aef1)
#8 isc_logconfig_create /home/ondrej/Projects/bind9/lib/isc/log.c:322 (libisc.so.1504+0x3035f)
#9 isc_logconfig_create /home/ondrej/Projects/bind9/lib/isc/log.c:313 (libisc.so.1504+0x3035f)
#10 load_configuration server.c:9091 (named+0x593a2)
#11 run_server server.c:9654 (named+0x59a47)
#12 dispatch /home/ondrej/Projects/bind9/lib/isc/task.c:1134 (libisc.so.1504+0x56fa6)
#13 run /home/ondrej/Projects/bind9/lib/isc/task.c:1319 (libisc.so.1504+0x56fa6)
#14 <null> <null> (libtsan.so.0+0x29b3d)
Location is heap block of size 89 at 0x7b18000059a0 allocated by thread T9:
#0 malloc <null> (libtsan.so.0+0x2b1a3)
#1 default_memalloc /home/ondrej/Projects/bind9/lib/isc/mem.c:685 (libisc.so.1504+0x33fee)
#2 mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:598 (libisc.so.1504+0x34c7e)
#3 mem_allocateunlocked /home/ondrej/Projects/bind9/lib/isc/mem.c:1222 (libisc.so.1504+0x34c7e)
#4 isc___mem_allocate /home/ondrej/Projects/bind9/lib/isc/mem.c:1242 (libisc.so.1504+0x34c7e)
#5 isc__mem_allocate /home/ondrej/Projects/bind9/lib/isc/mem.c:2387 (libisc.so.1504+0x3be64)
#6 isc___mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:1007 (libisc.so.1504+0x3c6ca)
#7 isc__mem_get /home/ondrej/Projects/bind9/lib/isc/mem.c:2365 (libisc.so.1504+0x3aef1)
#8 isc_logconfig_create /home/ondrej/Projects/bind9/lib/isc/log.c:322 (libisc.so.1504+0x3035f)
#9 isc_logconfig_create /home/ondrej/Projects/bind9/lib/isc/log.c:313 (libisc.so.1504+0x3035f)
#10 load_configuration server.c:9091 (named+0x593a2)
#11 run_server server.c:9654 (named+0x59a47)
#12 dispatch /home/ondrej/Projects/bind9/lib/isc/task.c:1134 (libisc.so.1504+0x56fa6)
#13 run /home/ondrej/Projects/bind9/lib/isc/task.c:1319 (libisc.so.1504+0x56fa6)
#14 <null> <null> (libtsan.so.0+0x29b3d)
Mutex M944203102091890528 is already destroyed.
Mutex M246707628766581536 is already destroyed.
Mutex M15 (0x7b4800000198) created at:
#0 pthread_mutex_init <null> (libtsan.so.0+0x2c5ad)
#1 isc__mutex_init /home/ondrej/Projects/bind9/lib/isc/pthreads/mutex.c:284 (libisc.so.1504+0x7bc06)
#2 mem_create /home/ondrej/Projects/bind9/lib/isc/mem.c:737 (libisc.so.1504+0x3a749)
#3 isc_mem_create /home/ondrej/Projects/bind9/lib/isc/mem.c:2358 (libisc.so.1504+0x3a749)
#4 main main.c:1512 (named+0x1a64b)
Thread T21 'isc-socket-3' (tid=1350, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2be1b)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/pthreads/thread.c:75 (libisc.so.1504+0x7bcc4)
#2 isc_socketmgr_create2 /home/ondrej/Projects/bind9/lib/isc/unix/socket.c:3817 (libisc.so.1504+0x74ce2)
#3 create_managers main.c:919 (named+0x1af64)
#4 setup main.c:1235 (named+0x1af64)
#5 main main.c:1515 (named+0x1af64)
Thread T9 'isc-worker0000' (tid=1327, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2be1b)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/pthreads/thread.c:75 (libisc.so.1504+0x7bcc4)
#2 isc_taskmgr_create /home/ondrej/Projects/bind9/lib/isc/task.c:1410 (libisc.so.1504+0x59d63)
#3 create_managers main.c:902 (named+0x1aeec)
#4 setup main.c:1235 (named+0x1aeec)
#5 main main.c:1515 (named+0x1aeec)
SUMMARY: ThreadSanitizer: data race /home/ondrej/Projects/bind9/lib/isc/log.c:1483 in isc_log_wouldlog
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1468Use after free in dns_dnssec_matchdskey2020-01-14T20:17:58ZMark AndrewsUse after free in dns_dnssec_matchdskeymkeys system test failed
Job [#463900](https://gitlab.isc.org/isc-projects/bind9/-/jobs/463900) failed for 067ab246fe96957d36a62d231b83494d1a1397e2:
Additionally the wrong path name was used.
```
I:mkeys:reinitialize trust anchors, a...mkeys system test failed
Job [#463900](https://gitlab.isc.org/isc-projects/bind9/-/jobs/463900) failed for 067ab246fe96957d36a62d231b83494d1a1397e2:
Additionally the wrong path name was used.
```
I:mkeys:reinitialize trust anchors, add second key to bind.keys
I:mkeys:no response from ns2
I:mkeys:failed
No test directory: "/builds/isc-projects/bind9/bin/tests/mkeys"
I:mkeys:ns2 died before a SIGTERM was sent
R:mkeys:FAIL
E:mkeys:Tue Dec 10 02:26:40 UTC 2019
```
```
=================================================================
==5224==
ERROR: AddressSanitizer: heap-use-after-free on address 0x6070003924e0 at pc 0x7f47c32ee88e bp 0x7f47b38ef840 sp 0x7f47b38eeff0
READ of size 36 at 0x6070003924e0 thread T14
#0 0x7f47c32ee88d (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xd588d)
#1 0x7f47c0ab66d1 in isc_region_compare /builds/isc-projects/bind9/lib/isc/region.c:31
#2 0x7f47c1ee65ed in compare_ds rdata/generic/ds_43.c:229
#3 0x7f47c1ee65ed in dns_rdata_compare /builds/isc-projects/bind9/lib/dns/rdata.c:637
#4 0x7f47c1c48991 in dns_dnssec_matchdskey /builds/isc-projects/bind9/lib/dns/dnssec.c:2313
#5 0x7f47c20b0991 in validate_dnskey /builds/isc-projects/bind9/lib/dns/validator.c:2051
#6 0x7f47c20bd842 in validator_start /builds/isc-projects/bind9/lib/dns/validator.c:3155
#7 0x7f47c0acaffe in dispatch /builds/isc-projects/bind9/lib/isc/task.c:1134
#8 0x7f47c0acaffe in run /builds/isc-projects/bind9/lib/isc/task.c:1319
#9 0x7f47c02f9fb6 in start_thread /build/glibc-suXNNi/glibc-2.29/nptl/pthread_create.c:486
#10 0x7f47bf8be2ce in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xfa2ce)
0x6070003924e0 is located 16 bytes inside of 69-byte region [0x6070003924d0,0x607000392515)
freed by thread T16 here:
#0 0x7f47c3320277 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x107277)
#1 0x7f47c0a5deab in default_memfree /builds/isc-projects/bind9/lib/isc/mem.c:713
#2 0x7f47c0a67914 in mem_put /builds/isc-projects/bind9/lib/isc/mem.c:627
#3 0x7f47c0a67914 in isc___mem_free /builds/isc-projects/bind9/lib/isc/mem.c:1343
#4 0x7f47c0a77010 in isc__mem_free /builds/isc-projects/bind9/lib/isc/mem.c:2408
#5 0x7f47c0a78b36 in isc___mem_put /builds/isc-projects/bind9/lib/isc/mem.c:1061
#6 0x7f47c0a73443 in isc__mem_put /builds/isc-projects/bind9/lib/isc/mem.c:2373
#7 0x7f47c1c81a18 in free_dslist /builds/isc-projects/bind9/lib/dns/keytable.c:153
#8 0x7f47c1c87ebe in update_keynode /builds/isc-projects/bind9/lib/dns/keytable.c:184
#9 0x7f47c1c87ebe in insert /builds/isc-projects/bind9/lib/dns/keytable.c:342
#10 0x7f47c1c881bd in dns_keytable_add /builds/isc-projects/bind9/lib/dns/keytable.c:373
#11 0x7f47c20f292c in trust_key /builds/isc-projects/bind9/lib/dns/zone.c:3967
#12 0x7f47c216663a in keyfetch_done /builds/isc-projects/bind9/lib/dns/zone.c:10348
#13 0x7f47c0acaffe in dispatch /builds/isc-projects/bind9/lib/isc/task.c:1134
#14 0x7f47c0acaffe in run /builds/isc-projects/bind9/lib/isc/task.c:1319
#15 0x7f47c02f9fb6 in start_thread /build/glibc-suXNNi/glibc-2.29/nptl/pthread_create.c:486
previously allocated by thread T9 here:
#0 0x7f47c3320628 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x107628)
#1 0x7f47c0a5df68 in default_memalloc /builds/isc-projects/bind9/lib/isc/mem.c:685
#2 0x7f47c0a656f8 in mem_get /builds/isc-projects/bind9/lib/isc/mem.c:598
#3 0x7f47c0a656f8 in mem_allocateunlocked /builds/isc-projects/bind9/lib/isc/mem.c:1222
#4 0x7f47c0a656f8 in isc___mem_allocate /builds/isc-projects/bind9/lib/isc/mem.c:1242
#5 0x7f47c0a75968 in isc__mem_allocate /builds/isc-projects/bind9/lib/isc/mem.c:2387
#6 0x7f47c0a75dd1 in isc___mem_get /builds/isc-projects/bind9/lib/isc/mem.c:1007
#7 0x7f47c0a72766 in isc__mem_get /builds/isc-projects/bind9/lib/isc/mem.c:2365
#8 0x7f47c1c86a43 in prepend_keynode /builds/isc-projects/bind9/lib/dns/keytable.c:260
#9 0x7f47c1c880fd in insert /builds/isc-projects/bind9/lib/dns/keytable.c:355
#10 0x7f47c1c881bd in dns_keytable_add /builds/isc-projects/bind9/lib/dns/keytable.c:373
#11 0x55f7f1fe6892 in process_key server.c:1029
#12 0x55f7f1fe730e in load_view_keys server.c:1081
#13 0x55f7f1fe91ec in configure_view_dnsseckeys server.c:1240
#14 0x55f7f200a910 in configure_view server.c:5347
#15 0x55f7f203d4d3 in load_configuration server.c:8932
#16 0x55f7f204497b in run_server server.c:9654
#17 0x7f47c0acaffe in dispatch /builds/isc-projects/bind9/lib/isc/task.c:1134
#18 0x7f47c0acaffe in run /builds/isc-projects/bind9/lib/isc/task.c:1319
#19 0x7f47c02f9fb6 in start_thread /build/glibc-suXNNi/glibc-2.29/nptl/pthread_create.c:486
Thread T14 created by T0 here:
#0 0x7f47c32529b2 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x399b2)
#1 0x7f47c0b357fe in isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:75
#2 0x7f47c0ad34c0 in isc_taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1410
#3 0x55f7f1fc8f6c in create_managers main.c:902
#4 0x55f7f1fc8f6c in setup main.c:1235
#5 0x55f7f1fc8f6c in main main.c:1515
#6 0x7f47bf7eabba in __libc_start_main ../csu/libc-start.c:308
Thread T16 created by T0 here:
#0 0x7f47c32529b2 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x399b2)
#1 0x7f47c0b357fe in isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:75
#2 0x7f47c0ad34c0 in isc_taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1410
#3 0x55f7f1fc8f6c in create_managers main.c:902
#4 0x55f7f1fc8f6c in setup main.c:1235
#5 0x55f7f1fc8f6c in main main.c:1515
#6 0x7f47bf7eabba in __libc_start_main ../csu/libc-start.c:308
Thread T9 created by T0 here:
#0 0x7f47c32529b2 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x399b2)
#1 0x7f47c0b357fe in isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:75
#2 0x7f47c0ad34c0 in isc_taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1410
#3 0x55f7f1fc8f6c in create_managers main.c:902
#4 0x55f7f1fc8f6c in setup main.c:1235
#5 0x55f7f1fc8f6c in main main.c:1515
#6 0x7f47bf7eabba in __libc_start_main ../csu/libc-start.c:308
SUMMARY: AddressSanitizer: heap-use-after-free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xd588d)
Shadow bytes around the buggy address:
0x0c0e8006a440: fd fd fa fa fa fa fd fd fd fd fd fd fd fd fd fa
0x0c0e8006a450: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fa fa
0x0c0e8006a460: fa fa fd fd fd fd fd fd fd fd fd fd fa fa fa fa
0x0c0e8006a470: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fd fd
0x0c0e8006a480: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
=>0x0c0e8006a490: fd fd fd fd fd fd fa fa fa fa fd fd[fd]fd fd fd
0x0c0e8006a4a0: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fd
0x0c0e8006a4b0: fd fa fa fa fa fa fd fd fd fd fd fd fd fd fd fd
0x0c0e8006a4c0: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fa fa
0x0c0e8006a4d0: fa fa fd fd fd fd fd fd fd fd fd fa fa fa fa fa
0x0c0e8006a4e0: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==5224==ABORTING
```
context
```
10-Dec-2019 02:25:39.905 sockmgr 0x610000000150 thread 6: watcher got message -2 for socket -1
10-Dec-2019 02:25:39.905 managed-keys-zone: Returned from key fetch in keyfetch_done() for '.': success
10-Dec-2019 02:25:39.905 managed-keys-zone: Verifying DNSKEY set for zone '.' using DS 18325/8: success
10-Dec-2019 02:25:39.905 dispatchmgr 0x61500000ff90: destroy_mgr_ok: shuttingdown=0, listnonempty=1, depool=128, rpool=0, dpool=128
10-Dec-2019 02:25:39.905 managed-keys-zone: Initializing automatic trust anchor management for zone '.'; DNSKEY ID 18325 is now trusted, waiving the normal 30-day waiting period.
=================================================================
10-Dec-2019 02:25:39.905 set_refreshkeytimer: managed-keys-zone: enter
10-Dec-2019 02:25:39.905 managed-keys-zone: next key refresh: 10-Dec-2019 02:25:49.905
10-Dec-2019 02:25:39.905 zone_settimer: managed-keys-zone: enter
10-Dec-2019 02:25:39.905 managed-keys-zone: Initializing automatic trust anchor management for zone '.'; DNSKEY ID 30370 is now trusted, waiving the normal 30-day waiting period.
==5224==ERROR: AddressSanitizer: heap-use-after-free on address 0x6070003924e0 at pc 0x7f47c32ee88e bp 0x7f47b38ef840 sp 0x7f47b38eeff0
10-Dec-2019 02:25:39.905 set_refreshkeytimer: managed-keys-zone: enter
10-Dec-2019 02:25:39.905 managed-keys-zone: next key refresh: 10-Dec-2019 02:25:49.905
10-Dec-2019 02:25:39.905 zone_settimer: managed-keys-zone: enter
READ of size 36 at 0x6070003924e0 thread T14
10-Dec-2019 02:25:39.905 zone_journal: managed-keys-zone: enter
10-Dec-2019 02:25:39.905 writing to journal
10-Dec-2019 02:25:39.905 del . 0 IN SOA . . 1 0 0 0 0
10-Dec-2019 02:25:39.905 del . 0 IN TYPE65533 \# 16 00000000000000000000000000000000
10-Dec-2019 02:25:39.905 add . 0 IN SOA . . 2 0 0 0 0
10-Dec-2019 02:25:39.905 add . 0 IN TYPE65533 \# 276 5DEF022D5DEF0223000000000101030803010001A419854D3F352C1E E25F7827850AFC3D710B34ED03A925BDED58CCF28F389BD6BCB0644D C40202AA0446869F8F02FC7FAFF744561CD41B16025A355BA4CAF2C0 DFE87B1A3DB0CFB35E9101C22584FF94B928257A998C04BCD4CA7A4B 1C011CDC609F645B319F474AC26834C3EBBE44614E077A8E5866F8BD 43376096463EFB8C220D07310FCB086927E69DCD8D5A757111E252C7 E49807FC780FD67378B320377E2B803ED7A4F5AF2AD23FED1EB567C8 B64531B4EA48FBC83B88697E2049895964095AA3B5630EAFCA385268 D51499FC541CCD00DD862C88667C25B1529BC9B30F919C032746218F 61A02E6AFD26AE1E491FC5E93AC86F748D8A8BD52290A729
10-Dec-2019 02:25:39.905 add . 0 IN TYPE65533 \# 276 5DEF022D5DEF0223000000000101030803010001DD9361ABDDFBFBC9 8A25C0D02EF3FDD33051D17956C6B2C805EF0D5C81AF0D934B3DE4E3 7C4425E17C66A6709B6A00282DF5E61EFF5BD3D627492119BC2047AE 368A35739C540D230AF5DC9B5E9F0CB26F914DC29CD5832DC4D56B02 B5B885CE20C6388332FA6385A3A937A173EBAFCAD5E40AD57BFC513F 3D8B2BCC78FEB01563D877738E1BCBF008A9B342945229F788B93F42 948A4A71E5C47934D5A761F050D3C9B7BA90E6F1E5F98B2D15226E4D 74BAE812806AB264FCDF2543BB22DB7EB3155564620D424B99E0A005 CB458075A8ACB0A8381D62F324009F3B9E83C64842005AC54C0CDAD0 6F2C8B4355F28554023EE9265F38786896359255C34A0F57
10-Dec-2019 02:25:39.905 zone_needdump: managed-keys-zone: enter
10-Dec-2019 02:25:39.905 zone_settimer: managed-keys-zone: enter
#0 0x7f47c32ee88d (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xd588d)
#1 0x7f47c0ab66d1 in isc_region_compare /builds/isc-projects/bind9/lib/isc/region.c:31
#2 0x7f47c1ee65ed in compare_ds rdata/generic/ds_43.c:229
#3 0x7f47c1ee65ed in dns_rdata_compare /builds/isc-projects/bind9/lib/dns/rdata.c:637
#4 0x7f47c1c48991 in dns_dnssec_matchdskey /builds/isc-projects/bind9/lib/dns/dnssec.c:2313
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Evan HuntEvan Hunthttps://gitlab.isc.org/isc-projects/bind9/-/issues/1463named (ns5) dies silently in mkeys test2020-01-14T12:28:31ZOndřej Surýnamed (ns5) dies silently in mkeys testThe `named.run` ends with:
```
06-Dec-2019 12:51:53.331 sockmgr 0x7fba32bca020 thread 0: watcher got message -5 for socket 512
06-Dec-2019 12:51:53.331 sockmgr 0x7fba32bca020 thread 0: watcher got message -2 for socket -1
06-Dec-2019 12:...The `named.run` ends with:
```
06-Dec-2019 12:51:53.331 sockmgr 0x7fba32bca020 thread 0: watcher got message -5 for socket 512
06-Dec-2019 12:51:53.331 sockmgr 0x7fba32bca020 thread 0: watcher got message -2 for socket -1
06-Dec-2019 12:51:53.331 managed-keys-zone: Verifying DNSKEY set for zone '.' using DS 262/8: success
06-Dec-2019 12:51:53.331 managed-keys-zone: Initializing automatic trust anchor management for zone '.'; DNSKEY ID 262 is now trusted, waiving the normal 30-day waiting period.
06-Dec-2019 12:51:53.331 validating ./DNSKEY: no DNSKEY matching DS
06-Dec-2019 12:51:53.331 set_refreshkeytimer: managed-keys-zone: enter
06-Dec-2019 12:51:53.331 managed-keys-zone: next key refresh: 06-Dec-2019 12:51:53.331
06-Dec-2019 12:51:53.331 zone_settimer: managed-keys-zone: enter
```
and the system test log is:
```
[...]
I:mkeys:check key refreshes are resumed after root servers become available (31)
rndc: connect failed: 10.53.0.5#9709: connection refused
I:mkeys:failed
[...]
I:mkeys:ns5 died before a SIGTERM was sent
R:mkeys:FAIL
E:mkeys:Fri Dec 6 12:52:00 UTC 2019
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Evan HuntEvan Hunt