BIND issueshttps://gitlab.isc.org/isc-projects/bind9/-/issues2019-11-26T17:28:09Zhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1119Make unit tests ThreadSanitizer clean2019-11-26T17:28:09ZOndřej SurýMake unit tests ThreadSanitizer cleanThere's couple of TSAN warnings related to unit tests.There's couple of TSAN warnings related to unit tests.December 2019 (9.11.14, 9.14.9, 9.15.7)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1027Make rndc ThreadSanitizer clean2019-11-26T17:28:02ZOndřej SurýMake rndc ThreadSanitizer cleanCurrently `rndc` fails with:
```
==================
WARNING: ThreadSanitizer: data race (pid=27100)
Read of size 4 at 0x0001032e66fc by main thread:
#0 main rndc.c:1000 (rndc:x86_64+0x100001aa1)
Previous write of size 4 at 0x000...Currently `rndc` fails with:
```
==================
WARNING: ThreadSanitizer: data race (pid=27100)
Read of size 4 at 0x0001032e66fc by main thread:
#0 main rndc.c:1000 (rndc:x86_64+0x100001aa1)
Previous write of size 4 at 0x0001032e66fc by thread T2:
#0 rndc_connected rndc.c:450 (rndc:x86_64+0x100003532)
#1 dispatch task.c:1128 (rndc:x86_64+0x10003f7e5)
#2 run task.c:1295 (rndc:x86_64+0x10003d168)
Location is global 'connects' at 0x0001032e66fc (rndc+0x00010006f6fc)
Thread T2 (tid=18346395, running) created by main thread at:
#0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x2a17d)
#1 isc_thread_create thread.c:63 (rndc:x86_64+0x100055056)
#2 isc_taskmgr_create task.c:1380 (rndc:x86_64+0x10003cf10)
#3 main rndc.c:935 (rndc:x86_64+0x10000165b)
SUMMARY: ThreadSanitizer: data race rndc.c:1000 in main
==================
==================
WARNING: ThreadSanitizer: data race (pid=27100)
Read of size 4 at 0x0001032e6700 by main thread:
#0 main rndc.c:1000 (rndc:x86_64+0x100001ab6)
Previous write of size 4 at 0x0001032e6700 by thread T2:
#0 rndc_senddone rndc.c:274 (rndc:x86_64+0x100003fb9)
#1 dispatch task.c:1128 (rndc:x86_64+0x10003f7e5)
#2 run task.c:1295 (rndc:x86_64+0x10003d168)
Location is global 'sends' at 0x0001032e6700 (rndc+0x00010006f700)
Thread T2 (tid=18346395, running) created by main thread at:
#0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x2a17d)
#1 isc_thread_create thread.c:63 (rndc:x86_64+0x100055056)
#2 isc_taskmgr_create task.c:1380 (rndc:x86_64+0x10003cf10)
#3 main rndc.c:935 (rndc:x86_64+0x10000165b)
SUMMARY: ThreadSanitizer: data race rndc.c:1000 in main
==================
==================
WARNING: ThreadSanitizer: data race (pid=27100)
Read of size 4 at 0x0001032e6704 by main thread:
#0 main rndc.c:1000 (rndc:x86_64+0x100001acb)
Previous write of size 4 at 0x0001032e6704 by thread T2:
#0 rndc_recvdone rndc.c:294 (rndc:x86_64+0x1000040c3)
#1 dispatch task.c:1128 (rndc:x86_64+0x10003f7e5)
#2 run task.c:1295 (rndc:x86_64+0x10003d168)
Location is global 'recvs' at 0x0001032e6704 (rndc+0x00010006f704)
Thread T2 (tid=18346395, running) created by main thread at:
#0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x2a17d)
#1 isc_thread_create thread.c:63 (rndc:x86_64+0x100055056)
#2 isc_taskmgr_create task.c:1380 (rndc:x86_64+0x10003cf10)
#3 main rndc.c:935 (rndc:x86_64+0x10000165b)
SUMMARY: ThreadSanitizer: data race rndc.c:1000 in main
==================
ThreadSanitizer: reported 3 warnings
```
Should be fairly easy to fix...December 2019 (9.11.14, 9.14.9, 9.15.7)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/898"forward" system test fails intermittently2020-01-10T08:43:42ZMichał Kępień"forward" system test fails intermittentlyThe last check in the test, "checking that priming queries are not forwarded", can rarely trigger a false positive. See https://gitlab.isc.org/isc-projects/bind9/-/jobs/181625 for an example. The problem is that `dig` may return before...The last check in the test, "checking that priming queries are not forwarded", can rarely trigger a false positive. See https://gitlab.isc.org/isc-projects/bind9/-/jobs/181625 for an example. The problem is that `dig` may return before `ns1` logs the priming query:
---
* `ns7/named.run`:
```
22-Feb-2019 06:51:21.336 client @0x7fa44003dc10 10.53.0.1#39681 (txt.example1): query (cache) 'txt.example1/TXT/IN' approved
...
22-Feb-2019 06:51:21.337 res 0x7fa455437020: priming
22-Feb-2019 06:51:21.337 fetch: ./NS
...
22-Feb-2019 06:51:21.337 socket 0x7fa45543a310 10.53.0.7#43075: bound
22-Feb-2019 06:51:21.337 socket 0x7fa45543a600 10.53.0.7#51763: bound
...
...
22-Feb-2019 06:51:21.337 sending packet to 10.53.0.1#8600
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22324
;; flags:; QUESTION: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
; COOKIE: 4d1e44ce6bd38bf7
;; QUESTION SECTION:
;. IN NS
22-Feb-2019 06:51:21.337 sending packet to 10.53.0.4#8600
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14354
;; flags: rd; QUESTION: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
; COOKIE: 2f66027d9adee561
;; QUESTION SECTION:
;txt.example1. IN TXT
...
22-Feb-2019 06:51:21.338 received packet from 10.53.0.4#8600
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14354
;; flags: qr rd ra; QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; COOKIE: 2f66027d9adee56118fc22405c6f9be94038c7c62a19ec15
;; QUESTION SECTION:
;txt.example1. IN TXT
;; ANSWER SECTION:
;txt.example1. 300 IN TXT "forwarded"
...
22-Feb-2019 06:51:21.338 client @0x7fa44003dc10 10.53.0.1#39681 (txt.example1): send
22-Feb-2019 06:51:21.338 client @0x7fa44003dc10 10.53.0.1#39681 (txt.example1): sendto
22-Feb-2019 06:51:21.338 client @0x7fa44003dc10 10.53.0.1#39681 (txt.example1): senddone
22-Feb-2019 06:51:21.338 client @0x7fa44003dc10 10.53.0.1#39681 (txt.example1): next
22-Feb-2019 06:51:21.338 client @0x7fa44003dc10 10.53.0.1#39681 (txt.example1): ns_client_detach: ref = 0
22-Feb-2019 06:51:21.338 client @0x7fa44003dc10 10.53.0.1#39681 (txt.example1): endrequest
...
22-Feb-2019 06:51:21.349 received packet from 10.53.0.1#8600
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22324
;; flags: qr aa; QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; COOKIE: 4d1e44ce6bd38bf769a6f34b5c6f9be9b821635b54edb75d
;; QUESTION SECTION:
;. IN NS
;; ANSWER SECTION:
;. 300 IN NS a.root-servers.nil.
;; ADDITIONAL SECTION:
;a.root-servers.nil. 300 IN A 10.53.0.1
```
* `ns1/named.run`:
```
22-Feb-2019 06:51:21.348 client @0x7f696800ee90 10.53.0.7#51763 (.): query './NS/IN' approved
```
---
Thus, if:
```
sent=`grep "10.53.0.7#.* (.): query '\./NS/IN' approved" ns1/named.run | wc -l`
```
manages to complete before that line appears in `ns1/named.run`, a false positive will be triggered.
@wpk, since you wrote the test and it is a fairly recent addition, please fix it? :)December 2019 (9.11.14, 9.14.9, 9.15.7)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1556Release Checklist for BIND 9.11.15, BIND 9.11.15-S1, BIND 9.14.10, BIND 9.15.82020-02-07T12:58:51ZMichał KępieńRelease Checklist for BIND 9.11.15, BIND 9.11.15-S1, BIND 9.14.10, BIND 9.15.8## Release Schedule
**Tagging Deadline:** Wednesday, January 15th, 2020
**Public Release:** Wednesday, January 22nd, 2020
## Release Checklist
## 2 Working Days Before the Tagging Deadline
- [x] ***(QA)*** Check whether all issues ...## Release Schedule
**Tagging Deadline:** Wednesday, January 15th, 2020
**Public Release:** Wednesday, January 22nd, 2020
## Release Checklist
## 2 Working Days Before the Tagging Deadline
- [x] ***(QA)*** Check whether all issues assigned to the release milestone are resolved[^1].
- [x] ***(QA)*** Ensure that there are no outstanding merge requests in the private repository[^1] (Subscription Edition only).
- [x] ***(QA)*** Ensure all merge requests marked for backporting have been indeed backported.
## Before the Tagging Deadline
- [x] ***(QA)*** Inform Support/Marketing of impending release (and give estimated release dates).
- [x] ***(QA)*** Check Perflab to ensure there has been no unexplained drop in performance for the versions being released.
- [x] ***(SwEng)*** Update API files for libraries with new version information.
- [x] ***(SwEng)*** Change software version and library versions in `configure.ac` (new major release only).
- [x] ***(SwEng)*** Rebuild `configure` using Autoconf on `docs.isc.org`.
- [x] ***(SwEng)*** Update `CHANGES`.
- [x] ***(SwEng)*** Update `CHANGES.SE` (Subscription Edition only).
- [x] ***(SwEng)*** Update `README.md`.
- [x] ***(SwEng)*** Update `version`.
- [x] ***(SwEng)*** Build documentation on `docs.isc.org`.
- [x] ***(QA)*** Check that all the above steps were performed correctly.
- [x] ***(QA)*** Check that the contents of release notes match the merge requests comprising the releases.
- [x] ***(QA)*** Check that the formatting is correct for text, PDF, and HTML versions of release notes.
- [x] ***(SwEng)*** Tag the releases[^2]. (Tags may only be pushed to the public repository for releases which are *not* security releases.)
- [x] ***(SwEng)*** If this is the first tag for a release (e.g. beta), create a release branch named `release_v9_X_Y` to allow development to continue on the maintenance branch whilst release engineering continues.
## Before the ASN Deadline (for ASN Releases) or the Public Release Date (for Regular Releases)
- [x] ***(QA)*** Verify GitLab CI results for the tags created and prepare a QA report for the releases to be published.
- [x] ***(QA)*** Request signatures for the tarballs, providing their location and checksums.
- [x] ***(Signers)*** Validate tarball checksums, sign tarballs, and upload signatures.
- [x] ***(QA)*** Verify tarball signatures and check tarball checksums again.
- [x] ***(Support)*** Pre-publish ASN and/or Subscription Edition tarballs so that packages can be built.
- [x] ***(QA)*** Build and test ASN and/or Subscription Edition packages.
- [x] ***(QA)*** Notify Support that the releases have been prepared.
- [x] ***(Support)*** Send out ASNs (if applicable).
## On the Day of Public Release
- [x] ***(Support)*** Wait for clearance from Security Officer to proceed with the public release (if applicable).
- [x] ***(Support)*** Place tarballs in public location on FTP site.
- [x] ***(Support)*** Publish links to downloads on ISC website.
- [x] ***(Support)*** Write release email to *bind-announce*.
- [x] ***(Support)*** Write email to *bind-users* (if a major release).
- [x] ***(Support)*** Update tickets in case of waiting support customers.
- [x] ***(QA)*** Build and test any outstanding private packages.
- [x] ***(QA)*** Build public packages (`*.deb`, RPMs).
- [x] ***(QA)*** Inform Marketing of the release.
- [x] ***(QA)*** Update the internal [BIND release dates wiki page](https://wiki.isc.org/bin/view/Main/BindReleaseDates) when public announcement has been made.
- [x] ***(Marketing)*** Post short note to Twitter.
- [x] ***(Marketing)*** Update [Wikipedia entry for BIND](https://en.wikipedia.org/wiki/BIND).
- [x] ***(Marketing)*** Write blog article (if a major release).
- [x] ***(QA)*** Ensure all new tags are annotated and signed.
- [x] ***(SwEng)*** Push tags for the published releases to the public repository.
- [x] ***(SwEng)*** Merge the automatically prepared `prep 9.X.Y` commit which updates `version` and documentation on the release branch into the relevant maintenance branch (`v9_X`).
[^1]: If not, use the time remaining until the tagging deadline to ensure all outstanding issues are either resolved or moved to a different milestone.
[^2]: Preferred command line: `git tag -u <DEVELOPER_KEYID> -a -s -m "BIND 9.X.Y[alphatag]" v9_X_Y[alphatag]`, where `[alphatag]` is an optional string such as `b1`, `rc1`, etc.January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Michał KępieńMichał Kępień2020-01-22https://gitlab.isc.org/isc-projects/bind9/-/issues/1555Code with pthread specific semantics is not portable2020-01-13T08:54:56ZOndřej SurýCode with pthread specific semantics is not portableWhen working on C11 thread library support, I found two cases where we use pthread specific semantics:
* Use of `void *` as a return type
* Use of `PTHREAD_MUTEX_INITIALIZER` instead of `isc_mutex_init()` callWhen working on C11 thread library support, I found two cases where we use pthread specific semantics:
* Use of `void *` as a return type
* Use of `PTHREAD_MUTEX_INITIALIZER` instead of `isc_mutex_init()` callJanuary 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1552"geoip-use-ecs" broken with libmaxminddb 1.4.0+2020-01-13T14:07:09ZMichał Kępień"geoip-use-ecs" broken with libmaxminddb 1.4.0+An [upstream code change][1] introduced in libmaxminddb 1.4.0 causes the `geoip2` system test on *v9_11* to consistently fail due to the libmaxminddb library apparently thinking that 127.0.0.1 matches the `::10.53.0.1/128` IP range:
```...An [upstream code change][1] introduced in libmaxminddb 1.4.0 causes the `geoip2` system test on *v9_11* to consistently fail due to the libmaxminddb library apparently thinking that 127.0.0.1 matches the `::10.53.0.1/128` IP range:
```
$ grep . bin/tests/system/geoip2/dig.out.ns2.test31.*
bin/tests/system/geoip2/dig.out.ns2.test31.1:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.2:"2"
bin/tests/system/geoip2/dig.out.ns2.test31.3:"3"
bin/tests/system/geoip2/dig.out.ns2.test31.4:"4"
bin/tests/system/geoip2/dig.out.ns2.test31.5:"5"
bin/tests/system/geoip2/dig.out.ns2.test31.6:"6"
bin/tests/system/geoip2/dig.out.ns2.test31.7:"7"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.1:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.2:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.3:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.4:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.5:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.6:"1"
bin/tests/system/geoip2/dig.out.ns2.test31.ecs.7:"1"
```
I know nothing about how libmaxminddb matches stuff internally, neither have I had the time to examine what the aforementioned upstream change does. This may even be an upstream bug in the library, but I would rather exercise caution before telling folks something they wrote does not work as intended when I am not 100% sure what those intentions are ;) To a novice like me, it looks like the library might now be treating an address from an unknown IP range as matching the first IP range defined in the database?
I tried rebuilding the `*.mmdb` files from `*.json` files in `bin/tests/system/geoip2/data/` using the `write-test-data.pl` script, but that did not change anything.
While this report only strictly applies to 9.11 because that is the only branch where the failing test still exists (`use-geoip-ecs` was ripped out in later releases), perhaps it is a symptom of an unexpected behavior which is *not* limited to our tests, but rather one that is occurring with newer versions of libmaxminddb in general (and we are simply unaware of it)?
@each, do you have any ideas on how to approach this?
[1]: https://github.com/maxmind/libmaxminddb/commit/2d49f4f04dad3103db701921258f3fac3b16ed80January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1547Out-of-tree build fails with uverr2result.c:15:10: fatal error: isc/platform....2020-01-09T09:24:41ZMichal NowakOut-of-tree build fails with uverr2result.c:15:10: fatal error: isc/platform.h: No such file or directoryOut-of-tree build fails with:
```
libtool: compile: gcc -include /builds/isc-projects/bind9/workspace/config.h -I/builds/isc-projects/bind9/workspace -I../../../.. -I../../../../lib/isc/netmgr/../include -I../../../../lib/isc/netmgr/../...Out-of-tree build fails with:
```
libtool: compile: gcc -include /builds/isc-projects/bind9/workspace/config.h -I/builds/isc-projects/bind9/workspace -I../../../.. -I../../../../lib/isc/netmgr/../include -I../../../../lib/isc/netmgr/../unix/include -I../../../../lib/isc/netmgr/../pthreads/include -I../../../../lib/isc/netmgr/.. -I/usr/include/json-c -I/usr/include/libxml2 -DISC_MEM_DEFAULTFILL=1 -DISC_LIST_CHECKINIT=1 -fno-omit-frame-pointer -fno-optimize-sibling-calls -O1 -g -Wall -Wextra -O3 -pthread -I/usr/include/google -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -Wno-missing-field-initializers -fno-strict-aliasing -Wshadow -Werror -c ../../../../lib/isc/netmgr/tcp.c -fPIC -DPIC -o .libs/tcp.o
../../../../lib/isc/netmgr/uverr2result.c:15:10: fatal error: isc/platform.h: No such file or directory
15 | #include <isc/platform.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [Makefile:185: uverr2result.lo] Error 1
In file included from ../../../../lib/isc/netmgr/../include/isc/buffer.h:105,
from ../../../../lib/isc/netmgr/udp.c:16:
../../../../lib/isc/netmgr/../include/isc/assertions.h:20:10: fatal error: isc/platform.h: No such file or directory
20 | #include <isc/platform.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [Makefile:185: udp.lo] Error 1
In file included from ../../../../lib/isc/netmgr/../include/isc/buffer.h:105,
from ../../../../lib/isc/netmgr/tcpdns.c:16:
../../../../lib/isc/netmgr/../include/isc/assertions.h:20:10: fatal error: isc/platform.h: No such file or directory
20 | #include <isc/platform.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
In file included from ../../../../lib/isc/netmgr/../include/isc/buffer.h:105,
from ../../../../lib/isc/netmgr/netmgr.c:17:
../../../../lib/isc/netmgr/../include/isc/assertions.h:20:10: fatal error: isc/platform.h: No such file or directory
20 | #include <isc/platform.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
In file included from ../../../../lib/isc/netmgr/../include/isc/buffer.h:105,
from ../../../../lib/isc/netmgr/tcp.c:17:
../../../../lib/isc/netmgr/../include/isc/assertions.h:20:10: fatal error: isc/platform.h: No such file or directory
20 | #include <isc/platform.h>
| ^~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [Makefile:185: tcpdns.lo] Error 1
make[3]: *** [Makefile:185: tcp.lo] Error 1
make[3]: *** [Makefile:185: netmgr.lo] Error 1
make[3]: Target 'all' not remade because of errors.
make[3]: Leaving directory '/builds/isc-projects/bind9/workspace/lib/isc/netmgr'
make[2]: *** [Makefile:212: subdirs] Error 1
```
We had a similar one recently in 9.14 branch: https://gitlab.isc.org/isc-projects/bind9/issues/1530.January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Michal NowakMichal Nowakhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1546Add out-of-tree build to CI2020-01-09T09:43:43ZMichal NowakAdd out-of-tree build to CIIt seems that all our CI build jobs build BIND while in it's sources, hence https://gitlab.isc.org/isc-projects/bind9/issues/1530 was present for some time and only identified when the BIND package was being updated in distributions, whi...It seems that all our CI build jobs build BIND while in it's sources, hence https://gitlab.isc.org/isc-projects/bind9/issues/1530 was present for some time and only identified when the BIND package was being updated in distributions, which happened to build out-of-tree.
We should add an out-of-tree build job to the CI.January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Michal NowakMichal Nowakhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1530lib/dns/gen.c:29:26: fatal error: isc/platform.h: No such file or directory2020-01-07T11:37:53ZMichal Nowaklib/dns/gen.c:29:26: fatal error: isc/platform.h: No such file or directoryIn OpenIndiana (an illumos distribution) we build BIND out-of-tree and BIND 9.14.9 fails to build for me with:
```
making all in /userland/ws/oi-userland/components/network/bind/build/amd64/lib/dns
make[3]: Entering directory '/userland/...In OpenIndiana (an illumos distribution) we build BIND out-of-tree and BIND 9.14.9 fails to build for me with:
```
making all in /userland/ws/oi-userland/components/network/bind/build/amd64/lib/dns
make[3]: Entering directory '/userland/ws/oi-userland/components/network/bind/build/amd64/lib/dns'
/usr/gcc/6/bin/gcc -m64 -O3 -D_XOPEN_SOURCE=600 -D__EXTENSIONS__=1 -D_XPG6 -D_POSIX_PTHREAD_SEMANTICS -pthread -I/usr/include/libxml2 -fPIC -I/userland/ws/oi-userland/components/network/bind/bind-9.14.9/lib/isc/include \
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 \
-m64 -Wl,-ztext -o gen /userland/ws/oi-userland/components/network/bind/bind-9.14.9/lib/dns/gen.c \
-lnsl -lsocket -lscf -ljson-c -llmdb -lxml2 -lz -llzma -lpthread -lm -lsocket -lnsl
/userland/ws/oi-userland/components/network/bind/bind-9.14.9/lib/dns/gen.c:29:26: fatal error: isc/platform.h: No such file or directory
#include <isc/platform.h>
^
```
The culprit seems to originated in d10fbdec84b5cbe2e51e0b74f41716dcc2c7035d.
I was able to make the problem away with the following patch:
```patch
--- bind-9.14.9/lib/dns/Makefile.in 2019-12-12 07:12:17.000000000 +0000
+++ bind-9.14.9/lib/dns/Makefile.in 2019-12-22 15:48:17.354544762 +0000
@@ -172,7 +172,7 @@ code.h: gen
./gen -s ${srcdir} > code.h || { rm -f $@ ; exit 1; }
gen: gen.c
- ${BUILD_CC} ${BUILD_CFLAGS} -I${top_srcdir}/lib/isc/include \
+ ${BUILD_CC} ${BUILD_CFLAGS} -I../../lib/isc/include \
${LFS_CFLAGS} ${LFS_LDFLAGS} \
${BUILD_CPPFLAGS} ${BUILD_LDFLAGS} -o $@ ${srcdir}/gen.c \
${BUILD_LIBS} ${LFS_LIBS}
```
If it's indeed an out-of-tree build issue, respective CI job might we worth.January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1525inline system test failed, need to wait for zone to be loaded.2020-01-14T13:56:57ZMark Andrewsinline system test failed, need to wait for zone to be loaded.Job [#493996](https://gitlab.isc.org/isc-projects/bind9/-/jobs/493996) failed for 41d827893e2b8529738491e4780fd76039ccf291:
```
I:inline:checking that records added from a journal are scheduled to be resigned (74)
I:inline:failed
I:inli...Job [#493996](https://gitlab.isc.org/isc-projects/bind9/-/jobs/493996) failed for 41d827893e2b8529738491e4780fd76039ccf291:
```
I:inline:checking that records added from a journal are scheduled to be resigned (74)
I:inline:failed
I:inline:check that zonestatus reports 'type: master' for a inline master zone (75)
I:inline:check that zonestatus reports 'type: slave' for a inline slave zone (76)
I:inline:exit status: 1
R:inline:FAIL
E:inline:Fri Dec 20 08:36:39 UTC 2019
```
```
[beetle:bin/tests/system] marka% !mo
more inline/rndc.out.ns3.post.test74
lt-rndc: 'zonestatus' failed: zone not loaded
[beetle:bin/tests/system] marka%
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1524stats.c:300:29:error: undeclared (first use in this function): stats->counter...2019-12-19T22:41:14ZMichal Nowakstats.c:300:29:error: undeclared (first use in this function): stats->counters[counter] = val;Build of BIND 9.11.14 on ppc64le CentOS 7 Copr build root fails with:
```
/bin/sh /builddir/build/BUILD/bind-9.11.14/libtool --mode=compile --tag=CC gcc -std=gnu99 -I/builddir/build/BUILD/bind-9.11.14 -I../.. -I./unix/include -I./pthrea...Build of BIND 9.11.14 on ppc64le CentOS 7 Copr build root fails with:
```
/bin/sh /builddir/build/BUILD/bind-9.11.14/libtool --mode=compile --tag=CC gcc -std=gnu99 -I/builddir/build/BUILD/bind-9.11.14 -I../.. -I./unix/include -I./pthreads/include -I./noatomic/include -I./include -I./include -I/builddir/build/BUILD/bind-9.11.14/lib/dns/include -I../../lib/dns/include -I/opt/isc/isc-bind/root/usr/include -D_REENTRANT -DOPENSSL -DPK11_LIB_LOCATION=\"undefined\" -D_GNU_SOURCE -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -I/usr/include/libxml2 -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -fno-strict-aliasing -fno-delete-null-pointer-checks -c stats.c
libtool: compile: gcc -std=gnu99 -I/builddir/build/BUILD/bind-9.11.14 -I../.. -I./unix/include -I./pthreads/include -I./noatomic/include -I./include -I./include -I/builddir/build/BUILD/bind-9.11.14/lib/dns/include -I../../lib/dns/include -I/opt/isc/isc-bind/root/usr/include -D_REENTRANT -DOPENSSL -DPK11_LIB_LOCATION=\"undefined\" -D_GNU_SOURCE -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -I/usr/include/libxml2 -fPIC -W -Wall -Wmissing-prototypes -Wcast-qual -Wwrite-strings -Wformat -Wpointer-arith -fno-strict-aliasing -fno-delete-null-pointer-checks -c stats.c -fPIC -DPIC -o .libs/stats.o
�[01m�[Kstats.c:�[m�[K In function '�[01m�[Ksetcounter�[m�[K':
�[01m�[Kstats.c:300:29:�[m�[K �[01;31m�[Kerror: �[m�[K'�[01m�[Kval�[m�[K' undeclared (first use in this function)
stats->counters[counter] = val;
�[01;32m�[K ^�[m�[K
�[01m�[Kstats.c:300:29:�[m�[K �[01;36m�[Knote: �[m�[Keach undeclared identifier is reported only once for each function it appears in
�[01m�[Kstats.c:286:20:�[m�[K �[01;35m�[Kwarning: �[m�[Kunused parameter '�[01m�[Kvalue�[m�[K' [-Wunused-parameter]
const uint64_t value)
�[01;32m�[K ^�[m�[K
make[2]: *** [stats.lo] Error 1
make[2]: Leaving directory `/builddir/build/BUILD/bind-9.11.14/lib/isc'
make[1]: *** [subdirs] Error 1
make[1]: Leaving directory `/builddir/build/BUILD/bind-9.11.14/lib'
make: *** [subdirs] Error 1
error: Bad exit status from /var/tmp/rpm-tmp.m7mn1J (%build)
```
See https://copr-be.cloud.fedoraproject.org/results/isc/bind-esv/srpm-builds/01131112/builder-live.log.gz (also attached as [builder-live.log](/uploads/04bea65eacd81485c149868a0b630355/builder-live.log)).
Recently changed in e50abe8a70e by @ondrej.January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1523pkcs11-destroy's usage message is misleading2019-12-20T09:06:34ZMichal Nowakpkcs11-destroy's usage message is misleading`pkcs11-destroy`'s usage message says "use whatever options you want, if any":
```
# /root/bind9/bin/pkcs11/pkcs11-destroy -h
Unrecognised option: -h
Usage:
pkcs11-destroy [-m module] [-s slot] [-i id | -l label] [-p pin] [-w waittime]
...`pkcs11-destroy`'s usage message says "use whatever options you want, if any":
```
# /root/bind9/bin/pkcs11/pkcs11-destroy -h
Unrecognised option: -h
Usage:
pkcs11-destroy [-m module] [-s slot] [-i id | -l label] [-p pin] [-w waittime]
```
But `pkcs11-destroy(8)` man page says that either `-i`, or `-l` needs to be provided:
```
SYNOPSIS
pkcs11-destroy [-m module] [-s slot] {-i ID | -l label} [-p PIN] [-w seconds]
DESCRIPTION
pkcs11-destroy destroys keys stored in a PKCS#11 device, identified by their ID or label.
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1513inline system test failed2020-01-08T07:32:00ZMark Andrewsinline system test failedJob [#485428](https://gitlab.isc.org/isc-projects/bind9/-/jobs/485428) failed for 68b9df92ac9217e06b0e30fb9f4e946385906040:
```
I:inline:testing updating dynamic serial via 'rndc signing -serial' (64)
I:inline:failed
I:inline:testing up...Job [#485428](https://gitlab.isc.org/isc-projects/bind9/-/jobs/485428) failed for 68b9df92ac9217e06b0e30fb9f4e946385906040:
```
I:inline:testing updating dynamic serial via 'rndc signing -serial' (64)
I:inline:failed
I:inline:testing updating dynamic serial via 'rndc signing -serial' with negative change (65)
```
```
[beetle:~/git/bind9/art] marka% cat bin/tests/system/inline/dig.out.ns2.pre.test64
; <<>> DiG 9.15.7 <<>> +tcp +dnssec -p 8600 bits. SOA @10.53.0.2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6398
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; COOKIE: d002aaae9aa2d2bb010000005df81fd0cd03a161bb275167 (good)
;; QUESTION SECTION:
;bits. IN SOA
;; ANSWER SECTION:
bits. 0 IN SOA ns2.bits. . 2011072460 20 20 1814400 3600
;; Query time: 8 msec
;; SERVER: 10.53.0.2#8600(10.53.0.2)
;; WHEN: Tue Dec 17 00:22:40 UTC 2019
;; MSG SIZE rcvd: 100
[beetle:~/git/bind9/art] marka%
```
```
[beetle:~/git/bind9/art] marka% cat bin/tests/system/inline/dig.out.ns2.post.test64
; <<>> DiG 9.15.7 <<>> +tcp +dnssec -p 8600 bits. SOA @10.53.0.2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12194
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; COOKIE: bcfc93baa244520a010000005df81fd1404c83bd457e385e (good)
;; QUESTION SECTION:
;bits. IN SOA
;; ANSWER SECTION:
bits. 0 IN SOA ns2.bits. . 2011072460 20 20 1814400 3600
;; Query time: 8 msec
;; SERVER: 10.53.0.2#8600(10.53.0.2)
;; WHEN: Tue Dec 17 00:22:41 UTC 2019
;; MSG SIZE rcvd: 100
[beetle:~/git/bind9/art] marka%
```
```
17-Dec-2019 00:22:40.182 received control channel command 'signing -serial 2011072470 bits'
17-Dec-2019 00:22:40.182 socket 0x7f358802d008: socket_recv: event 0x7f35c5273440 -> task 0x7f35c5272020
17-Dec-2019 00:22:40.182 setserial: zone bits/IN: enter
17-Dec-2019 00:22:40.182 zone bits/IN: could not get zone keys for secure dynamic update
17-Dec-2019 00:22:40.182 zone_journal: zone bits/IN: enter
17-Dec-2019 00:22:40.182 writing to journal
17-Dec-2019 00:22:40.182 del bits. 0 IN SOA ns2.bits. . 2011072460 20 20 1814400 3600
17-Dec-2019 00:22:40.182 add bits. 0 IN SOA ns2.bits. . 2011072470 20 20 1814400 3600
17-Dec-2019 00:22:40.182 sockmgr 0x7f35c5260020 thread 0: watcher got message -3 for socket 64
17-Dec-2019 00:22:40.182 sockmgr 0x7f35c5260020 thread 0: watcher got message -2 for socket -1
17-Dec-2019 00:22:40.182 socket 0x7f358802d008: internal_recv: event 0x7f35c5273440 -> task 0x7f35c5272020
17-Dec-2019 00:22:41.838 clientmgr @0x7f35c51ed020: clientmctx
17-Dec-2019 00:22:41.838 clientmgr @0x7f35c51ed020 attach: 18
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (<unknown-query>): query_reset
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 (no-peer): ns_client_newnamebuf
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 (no-peer): ns_client_newnamebuf: done
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 (no-peer): client_setup
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 (no-peer): allocate new client
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369: TCP request
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369: using view '_default'
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369: request is not signed
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369: recursion not available
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369: query
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (<unknown-query>): ns_query_start
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): qctx_init
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): client attr:0x22311, query attr:0x700, restarts:0, origqname:bits, timer:0, authdb:0, referral:0
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): ns__query_start
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): query 'bits/SOA/IN' approved
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_lookup
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): ns_client_getnamebuf
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): ns_client_getnamebuf: done
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): ns_client_newname
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): ns_client_newname: done
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_gotanswer
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): rrl=(nil), HAVECOOKIE=0, result=ISC_R_SUCCESS, fname=0x7f358c01a2e0(1), is_zone=1, RECURSIONOK=0, query.rpz_st=(nil)(0), RRL_CHECKED=0
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_checkrpz
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): rpz_rewrite
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_prepresponse
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_zerottl_refetch
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_respond
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_getexpire
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_addanswer
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_addrrset
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): ns_client_keepname
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_setorder
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_additional
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_additional: done
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_addrrset: done
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_addnoqnameproof
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_addauth
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): ns_query_done
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): send
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): senddone
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): reset client
17-Dec-2019 00:22:41.838 client @0x7f358c057bb0 10.53.0.1#53369 (bits): endrequest
17-Dec-2019 00:22:41.838 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (bits/SOA): query_reset
17-Dec-2019 00:22:41.846 client @0x7f358c057bb0 10.53.0.1#53369: freeing client
17-Dec-2019 00:22:42.038 query client=0x7f358c057bb0 thread=0x7f35c04d8700 (<unknown-query>): query_reset
17-Dec-2019 00:22:42.038 clientmgr @0x7f35c51ed020 detach: 17
17-Dec-2019 00:22:42.046 zone_needdump: zone bits/IN: enter
17-Dec-2019 00:22:42.046 zone_settimer: zone bits/IN: enter
17-Dec-2019 00:22:42.078 socket 0x7f358802d008: destroying
17-Dec-2019 00:22:42.078 sockmgr 0x7f35c5260020 thread 0: watcher got message -5 for socket 64
17-Dec-2019 00:22:42.078 sockmgr 0x7f35c5260020 thread 0: watcher got message -2 for socket -1
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/1507ThreadSanitizer: data race - lib/dns/tests/zt_test2020-01-13T11:11:14ZMark AndrewsThreadSanitizer: data race - lib/dns/tests/zt_testlib/dns/tests/zt_test failed
```
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by main thread (mutexes: write M1, write M2):
#0 free <null>
#1 default_memfree lib/isc/mem.c:713:2
#2 mem_put lib/isc...lib/dns/tests/zt_test failed
```
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by main thread (mutexes: write M1, write M2):
#0 free <null>
#1 default_memfree lib/isc/mem.c:713:2
#2 mem_put lib/isc/mem.c:627:2
#3 isc___mem_putanddetach lib/isc/mem.c:960:3
#4 isc__mem_putanddetach lib/isc/mem.c:2380:2
#5 zt_destroy lib/dns/zt.c:233:2
#6 zt_flushanddetach lib/dns/zt.c:250:3
#7 dns_zt_detach lib/dns/zt.c:261:2
#8 view_flushanddetach lib/dns/view.c:594:5
#9 dns_view_detach lib/dns/view.c:635:2
#10 asyncload_zt lib/dns/tests/zt_test.c:339:2
#11 <null> <null>
#12 __libc_start_main <null>
Previous atomic write of size 8 at 0x000000000001 by thread T1:
#0 __tsan_atomic64_fetch_sub <null>
#1 doneloading lib/dns/zt.c:551:6
#2 zone_asyncload lib/dns/zone.c:2201:3
#3 dispatch lib/isc/task.c:1134:7
#4 run lib/isc/task.c:1319:2
Mutex M1 (0x000000000020) created at:
#0 pthread_mutex_init <null>
#1 isc__mutex_init lib/isc/pthreads/mutex.c:284:8
#2 dns_view_create lib/dns/view.c:107:2
#3 dns_test_makeview lib/dns/tests/dnstest.c:225:2
#4 dns_test_makezone lib/dns/tests/dnstest.c:273:12
#5 asyncload_zt lib/dns/tests/zt_test.c:275:11
#6 <null> <null>
#7 __libc_start_main <null>
Mutex M2 (0x000000000027) created at:
#0 pthread_mutex_init <null>
#1 isc__mutex_init lib/isc/pthreads/mutex.c:284:8
#2 mem_create lib/isc/mem.c:737:2
#3 isc_mem_create lib/isc/mem.c:2358:2
#4 dns_test_begin lib/dns/tests/dnstest.c:145:2
#5 _setup lib/dns/tests/zt_test.c:55:11
#6 <null> <null>
#7 __libc_start_main <null>
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:75:8
#2 isc_taskmgr_create lib/isc/task.c:1410:3
#3 create_managers lib/dns/tests/dnstest.c:119:2
#4 dns_test_begin lib/dns/tests/dnstest.c:180:3
#5 _setup lib/dns/tests/zt_test.c:55:11
#6 <null> <null>
#7 __libc_start_main <null>
SUMMARY: ThreadSanitizer: data race in free
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/1500Be more verbose when opening IPC socket or fsyncing dir fails2020-01-13T21:50:40ZWitold KrecickiBe more verbose when opening IPC socket or fsyncing dir failsJanuary 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Witold KrecickiWitold Krecickihttps://gitlab.isc.org/isc-projects/bind9/-/issues/1499Deadlock when TCP listening fails2020-01-13T21:50:38ZWitold KrecickiDeadlock when TCP listening failsJanuary 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Witold KrecickiWitold Krecickihttps://gitlab.isc.org/isc-projects/bind9/-/issues/1497ThreadSanitizer: data race lib/isc/unix/socket.c:1839:2 in destroy2019-12-12T14:14:48ZOndřej SurýThreadSanitizer: data race lib/isc/unix/socket.c:1839:2 in destroy```
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1:
#0 destroy lib/isc/unix/socket.c:1839:2
#1 process_fd lib/isc/unix/socket.c:3177:3
#2 process_fds lib/isc/unix/socket.c:3257:3
#3 ne...```
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1:
#0 destroy lib/isc/unix/socket.c:1839:2
#1 process_fd lib/isc/unix/socket.c:3177:3
#2 process_fds lib/isc/unix/socket.c:3257:3
#3 netthread lib/isc/unix/socket.c:3503:10
Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M1, write M2):
#0 send_recvdone_event lib/isc/unix/socket.c
#1 isc_socket_cancel lib/isc/unix/socket.c:5053:5
#2 dns_dispatch_detach lib/dns/dispatch.c:2985:4
#3 fctx_cancelquery lib/dns/resolver.c:1378:3
#4 rctx_done lib/dns/resolver.c:9478:3
#5 resquery_response lib/dns/resolver.c
#6 dispatch lib/isc/task.c:1136:7
#7 run lib/isc/task.c:1323:2
Location is heap block of size 361 at 0x000000000014 allocated by thread T2:
#0 malloc <null>
#1 default_memalloc lib/isc/mem.c:685:8
#2 mem_get lib/isc/mem.c:598:8
#3 isc___mem_get lib/isc/mem.c:1013:9
#4 isc__mem_get lib/isc/mem.c:2365:10
#5 allocate_socket lib/isc/unix/socket.c:1876:9
#6 socket_create lib/isc/unix/socket.c:2450:11
#7 isc_socket_create lib/isc/unix/socket.c:2535:10
#8 fctx_query lib/dns/resolver.c:2044:12
#9 rctx_resend lib/dns/resolver.c:9376:11
#10 rctx_done lib/dns/resolver.c:9497:3
#11 resquery_response lib/dns/resolver.c
#12 dispatch lib/isc/task.c:1136:7
#13 run lib/isc/task.c:1323:2
Mutex M1 is already destroyed.
Mutex M2 (0x000000000026) created at:
#0 pthread_mutex_init <null>
#1 isc__mutex_init lib/isc/pthreads/mutex.c:284:8
#2 allocate_socket lib/isc/unix/socket.c:1913:2
#3 socket_create lib/isc/unix/socket.c:2450:11
#4 isc_socket_create lib/isc/unix/socket.c:2535:10
#5 fctx_query lib/dns/resolver.c:2044:12
#6 rctx_resend lib/dns/resolver.c:9376:11
#7 rctx_done lib/dns/resolver.c:9497:3
#8 resquery_response lib/dns/resolver.c
#9 dispatch lib/isc/task.c:1136:7
#10 run lib/isc/task.c:1323:2
Thread T2 'isc-socket-0' (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:75:8
#2 isc_socketmgr_create2 lib/isc/unix/socket.c:3817:3
#3 isc_socketmgr_create lib/isc/unix/socket.c:3774:10
#4 isc_socketmgr_createinctx lib/isc/unix/socket.c:5561:11
#5 main bin/delv/delv.c:1731:2
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:75:8
#2 isc_taskmgr_create lib/isc/task.c:1414:3
#3 isc_taskmgr_createinctx lib/isc/task.c:1982:11
#4 main bin/delv/delv.c:1730:2
SUMMARY: ThreadSanitizer: data race lib/isc/unix/socket.c:1839:2 in destroy
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Ondřej SurýOndřej Surýhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1492Backports missing from December maintenance releases2020-01-10T10:22:03ZMichał KępieńBackports missing from December maintenance releasesHere is a list[^1] of merge requests (along with their branch names) which, according to my brief examination, should be backported from *master* to *v9_14* and *v9_11*, but have not yet been backported. Almost all of them only fix test...Here is a list[^1] of merge requests (along with their branch names) which, according to my brief examination, should be backported from *master* to *v9_14* and *v9_11*, but have not yet been backported. Almost all of them only fix tests and thus are not critical, but there is one outlier (**in bold**) about which I am not sure:
- [x] !2750 `michal/fix-the-forward-system-test-on-windows`
- [x] !2747 `1479-_wait_for_rcode-adds-extraneous-query`
- [x] !2736 `ondrej/run-full-pipeline-on-schedule`
- [x] !2712 `1453-the-zero-system-test-timeouts-intermittently`
- [x] !2716 `1458-intermittent-failure-in-the-forward-system-test`
- [x] !2714 `1425-intermittent-failure-in-the-addzone-system-test`
- [x] !2715 `1401-intermittent-failures-in-the-catz-system-test`
- [x] !2705 `1427-intermittent-failure-in-fetchlimit-system-test`
- [x] !2704 `1407-intermittent-failure-in-the-mkeys-system-test`
- [x] !2671 `1410-intermittent-failure-in-the-resolver-test`
- [x] !2669 `1431-summary-sanitizer-grep-is-dangerous`
- [x] !2657 `ondrej/keep-sanitizer-tainted-system-tests-files`
- [x] !2649 `ondrej/stop-retrying-system-tests`
- [x] !2633 `1401-intermittent-failures-in-the-catz-system-test`
- [x] !2636 `1402-multiple-issues-in-the-runtime-system-test`
- [x] !2634 `ondrej/detect-cores-in-system-tests-on-FreeBSD`
- [x] !2601 `1380-autosign-jitter-test-fails-with-no-nsec3param-found-in-axfr`
- [x] !2554 `1256-fix-the-jitter-test-2`
- [x] !2553 `1256-fix-the-jitter-test`
- [x] !2611 **`1394-incoming-zone-transfer-messages-logged-to-wrong-category`**
- [x] !2602 `ondrej/get-the-backtraces-out-of-system-test-coredumps`
The MR **in bold** is both a bug fix and a functional change, so we should clarify whether it should be backported or not.
I made a list of checkboxes for all the other ones so that we can conveniently track the outstanding ones as I would like all test-related MRs (or at least those which apply relatively cleanly) to be backported for the January releases.
[^1]: in reverse merge order, i.e. starting from the most recently merged oneJanuary 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)Michał KępieńMichał Kępieńhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1486ThreadSanitizer: lock-order-inversion (potential deadlock) - dns_resolver_cre...2019-12-13T20:52:32ZOndřej SurýThreadSanitizer: lock-order-inversion (potential deadlock) - dns_resolver_createfetch vs dns_resolver_shutdown```
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=25300)
Cycle in lock order graph: M1131 (0x7b7400000238) => M1728 (0x7b4c000001d0) => M1131
Mutex M1728 acquired here while holding mutex M1131 in thread T...```
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=25300)
Cycle in lock order graph: M1131 (0x7b7400000238) => M1728 (0x7b4c000001d0) => M1131
Mutex M1728 acquired here while holding mutex M1131 in thread T1:
#0 pthread_mutex_lock <null> (delv+0x4492a6)
#1 fcount_incr /home/ondrej/Projects/bind9/lib/dns/resolver.c:1524:2 (libdns.so.1505+0x15b6ab)
#2 fctx_create /home/ondrej/Projects/bind9/lib/dns/resolver.c:4928:11 (libdns.so.1505+0x156959)
#3 dns_resolver_createfetch /home/ondrej/Projects/bind9/lib/dns/resolver.c:10588:12 (libdns.so.1505+0x15435e)
#4 create_fetch /home/ondrej/Projects/bind9/lib/dns/validator.c:1055:10 (libdns.so.1505+0x1a239a)
#5 seek_dnskey /home/ondrej/Projects/bind9/lib/dns/validator.c:1305:12 (libdns.so.1505+0x1a2f84)
#6 validate_answer /home/ondrej/Projects/bind9/lib/dns/validator.c:1557:13 (libdns.so.1505+0x1a06b3)
#7 validator_start /home/ondrej/Projects/bind9/lib/dns/validator.c:3157:13 (libdns.so.1505+0x19f013)
#8 dispatch /home/ondrej/Projects/bind9/lib/isc/task.c:1134:7 (libisc.so.1504+0x52007)
#9 run /home/ondrej/Projects/bind9/lib/isc/task.c:1319:2 (libisc.so.1504+0x4f3a8)
Mutex M1131 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (delv+0x4492a6)
#1 dns_resolver_createfetch /home/ondrej/Projects/bind9/lib/dns/resolver.c:10545:2 (libdns.so.1505+0x1540f4)
#2 create_fetch /home/ondrej/Projects/bind9/lib/dns/validator.c:1055:10 (libdns.so.1505+0x1a239a)
#3 seek_dnskey /home/ondrej/Projects/bind9/lib/dns/validator.c:1305:12 (libdns.so.1505+0x1a2f84)
#4 validate_answer /home/ondrej/Projects/bind9/lib/dns/validator.c:1557:13 (libdns.so.1505+0x1a06b3)
#5 validator_start /home/ondrej/Projects/bind9/lib/dns/validator.c:3157:13 (libdns.so.1505+0x19f013)
#6 dispatch /home/ondrej/Projects/bind9/lib/isc/task.c:1134:7 (libisc.so.1504+0x52007)
#7 run /home/ondrej/Projects/bind9/lib/isc/task.c:1319:2 (libisc.so.1504+0x4f3a8)
Mutex M1131 acquired here while holding mutex M1728 in main thread:
#0 pthread_mutex_lock <null> (delv+0x4492a6)
#1 dns_resolver_shutdown /home/ondrej/Projects/bind9/lib/dns/resolver.c:10312:4 (libdns.so.1505+0x155107)
#2 view_flushanddetach /home/ondrej/Projects/bind9/lib/dns/view.c:582:4 (libdns.so.1505+0x1a7031)
#3 dns_view_detach /home/ondrej/Projects/bind9/lib/dns/view.c:635:2 (libdns.so.1505+0x1a72eb)
#4 destroyclient /home/ondrej/Projects/bind9/lib/dns/client.c:611:3 (libdns.so.1505+0x213882)
#5 dns_client_destroy /home/ondrej/Projects/bind9/lib/dns/client.c:652:3 (libdns.so.1505+0x21369b)
#6 main /home/ondrej/Projects/bind9/bin/delv/delv.c:1827:3 (delv+0x4b9926)
Mutex M1728 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (delv+0x4492a6)
#1 dns_resolver_shutdown /home/ondrej/Projects/bind9/lib/dns/resolver.c:10307:2 (libdns.so.1505+0x155004)
#2 view_flushanddetach /home/ondrej/Projects/bind9/lib/dns/view.c:582:4 (libdns.so.1505+0x1a7031)
#3 dns_view_detach /home/ondrej/Projects/bind9/lib/dns/view.c:635:2 (libdns.so.1505+0x1a72eb)
#4 destroyclient /home/ondrej/Projects/bind9/lib/dns/client.c:611:3 (libdns.so.1505+0x213882)
#5 dns_client_destroy /home/ondrej/Projects/bind9/lib/dns/client.c:652:3 (libdns.so.1505+0x21369b)
#6 main /home/ondrej/Projects/bind9/bin/delv/delv.c:1827:3 (delv+0x4b9926)
Thread T1 'isc-worker0000' (tid=25315, running) created by main thread at:
#0 pthread_create <null> (delv+0x42b04b)
#1 isc_thread_create /home/ondrej/Projects/bind9/lib/isc/pthreads/thread.c:75:8 (libisc.so.1504+0x6a9ea)
#2 isc_taskmgr_create /home/ondrej/Projects/bind9/lib/isc/task.c:1410:3 (libisc.so.1504+0x4f1b2)
#3 isc_taskmgr_createinctx /home/ondrej/Projects/bind9/lib/isc/task.c:1978:11 (libisc.so.1504+0x51a92)
#4 main /home/ondrej/Projects/bind9/bin/delv/delv.c:1730:2 (delv+0x4b93d9)
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/home/ondrej/Projects/bind9/bin/delv/.libs/delv+0x4492a6) in pthread_mutex_lock
```January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)https://gitlab.isc.org/isc-projects/bind9/-/issues/1482autosign system test failed2019-12-13T08:58:19ZMark Andrewsautosign system test failedJob [#467197](https://gitlab.isc.org/isc-projects/bind9/-/jobs/467197) failed for 1afc984ba892ba9ae19051f00203b105299d21ed:Job [#467197](https://gitlab.isc.org/isc-projects/bind9/-/jobs/467197) failed for 1afc984ba892ba9ae19051f00203b105299d21ed:January 2020 (9.11.15, 9.14.10, 9.15.8, 9.11.15-S)