BIND issueshttps://gitlab.isc.org/isc-projects/bind9/-/issues2023-07-25T11:29:12Zhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4089[CVE-2023-2911] Exceeding the recursive-clients quota may cause named to term...2023-07-25T11:29:12ZDarren Ankney[CVE-2023-2911] Exceeding the recursive-clients quota may cause named to terminate unexpectedly when stale-answer-client-timeout is set to 0| Quick Links | :link: |
| ------------------------ | ---------------------------------------------------------------------------- |
| Incident Manager: ...| Quick Links | :link: |
| ------------------------ | ---------------------------------------------------------------------------- |
| Incident Manager: | @tkrizek |
| Deputy Incident Manager: | @ebf |
| Public Disclosure Date: | 2023-06-21 |
| CVSS Score: | [7.5][cvss_score] |
| Security Advisory: | https://gitlab.isc.org/isc-private/printing-press/-/merge_requests/56 |
| Mattermost Channel: | [CVE-2023-2911: crash with "stale-answer-client-timeout 0;"][mattermost_url] |
| Support Ticket: | N/A |
| Release Checklist: | https://gitlab.isc.org/isc-projects/bind9/-/issues/4123 |
| Post-mortem Etherpad: | [postmortem-2023-06][postmortem_url] |
[cvss_score]: https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H&version=3.1
[mattermost_url]: https://mattermost.isc.org/isc/channels/cve-2023-2911-crash-with-stale-answer-client-timeout-0
[postmortem_url]: https://pad.isc.org/p/postmortem-2023-06
:bulb: **Click [here][checklist_explanations] (internal resource) for general information about the security incident handling process.**
[checklist_explanations]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations
### Earlier Than T-5
- [x] [:link:][step_deputy] **(IM)** Pick a Deputy Incident Manager
- [x] ~~[:link:][step_respond] **(IM)** Respond to the bug reporter~~
- [x] [:link:][step_etherpad] **(IM)** [Create an Etherpad for post-mortem][postmortem_url]
- [x] [:link:][step_public_mrs] **(SwEng)** Ensure there are no public merge requests which inadvertently disclose the issue
- [x] [:link:][step_assign_cve_id] **(IM)** [Assign a CVE identifier](#note_376591)
- [x] [:link:][step_note_cve_info] **(SwEng)** [Update this issue with the assigned CVE identifier and the CVSS score](#note_376590)
- [x] [:link:][step_versions_affected] **(SwEng)** [Determine the range of product versions affected (including the Subscription Edition)](https://gitlab.isc.org/isc-projects/bind9/-/issues/4089#note_377894)
- [x] [:link:][step_workarounds] **(SwEng)** [Determine whether workarounds for the problem exist](https://gitlab.isc.org/isc-projects/bind9/-/issues/4089#note_378112)
- [x] ~~[:link:][step_coordinate] **(SwEng)** If necessary, coordinate with other parties~~
- [x] [:link:][step_earliest] **(Support)** Prepare and send out "earliest" notifications
- [x] [:link:][step_advisory_mr] **(Support)** [Create a merge request for the Security Advisory and include all readily available information in it](https://gitlab.isc.org/isc-private/printing-press/-/merge_requests/56)
- [x] [:link:][step_reproducer_mr] **(SwEng)** [Prepare a private merge request containing a system test reproducing the problem](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/522)
- [x] [:link:][step_notify_support] **(SwEng)** [Notify Support when a reproducer is ready](https://mattermost.isc.org/isc/pl/hhnzxhcyti8ei8q4t4nahn7f1r)
- [x] [:link:][step_code_analysis] **(SwEng)** [Prepare a detailed explanation of the code flow triggering the problem](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/523/diffs?commit_id=cd19fb673459b5e6dfda7bfee03a10c9b202eeb6)
- [x] [:link:][step_fix_mr] **(SwEng)** [Prepare a private merge request with the fix](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/523)
- [x] [:link:][step_review_fix] **(SwEng)** Ensure the merge request with the fix is reviewed and has no outstanding discussions
- [x] [:link:][step_review_docs] **(Support)** [Review the documentation changes introduced by the merge request with the fix](https://mattermost.isc.org/isc/pl/crtthy97ki817yeqti8fa7mfyo)
- [x] [:link:][step_backports] **(SwEng)** [Prepare backports of the merge request addressing the problem for all affected (and still maintained) branches of a given product](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/533)
- [x] [:link:][step_finish_advisory] **(Support)** [Finish preparing the Security Advisory](https://mattermost.isc.org/isc/pl/nt6beetinpnwzg8678sb5fi79r)
- [x] [:link:][step_meta_issue] **(QA)** [Create (or update) the private issue containing links to fixes & reproducers for all CVEs fixed in a given release cycle](https://gitlab.isc.org/isc-private/bind9/-/issues/68)
- [x] [:link:][step_changes] **(QA)** (BIND 9 only) Reserve a block of `CHANGES` placeholders once the complete set of vulnerabilities fixed in a given release cycle is determined
- [x] [:link:][step_merge_fixes] **(QA)** Merge the CVE fixes in CVE identifier order
- [x] [:link:][step_patches] **(QA)** [Prepare a standalone patch for the last stable release of each affected (and still maintained) product branch](https://gitlab.isc.org/isc-private/bind9/-/issues/68)
- [x] [:link:][step_asn_releases] **(QA)** [Prepare ASN releases (as outlined in the Release Checklist)](https://mattermost.isc.org/isc/pl/gbe5fz3bypfixqt67b85tapruc)
### At T-5
- [x] [:link:][step_send_asn] **(Support)** [Send ASN to eligible customers](https://mattermost.isc.org/isc/pl/43d9p7bou7g79e1zbnp91fooxr)
- [x] [:link:][step_preannouncement] **(Support)** [(BIND 9 only) Send a pre-announcement email to the *bind-announce* mailing list to alert users that the upcoming release will include security fixes](isc-private/printing-press!57)
### At T-4
- [x] [:link:][step_verify_asn] **(Support)** Verify that all ASN-eligible customers have received the notification email
### At T-1
- [x] [:link:][step_check_customers] **(Support)** Verify that any new or reinstated customers have received the notification email
- [x] [:link:][step_packager_emails] **(First IM)** [Send notifications to OS packagers](https://gitlab.isc.org/isc-private/printing-press/-/merge_requests/58#note_382746)
### On the Day of Public Disclosure
- [x] [:link:][step_clearance] **(IM)** [Grant Support clearance to proceed with public release](https://mattermost.isc.org/isc/pl/q9pwtbh6opbripwdf4inoxgsmo)
- [x] [:link:][step_publish] **(Support)** Publish the releases (as outlined in the release checklist)
- [x] [:link:][step_matrix] **(Support)** (BIND 9 only) Update vulnerability matrix in the Knowledge Base
- [x] [:link:][step_publish_advisory] **(Support)** Bump Document Version for the Security Advisory and publish it in the Knowledge Base
- [x] [:link:][step_notifications] **(First IM)** [Send notification emails to third parties](https://gitlab.isc.org/isc-private/printing-press/-/merge_requests/61#note_383415)
- [x] [:link:][step_mitre] **(First IM)** [Advise MITRE about the disclosed CVEs](https://gitlab.isc.org/isc-private/printing-press/-/merge_requests/56#note_383437)
- [x] [:link:][step_merge_advisory] **(First IM)** Merge the Security Advisory merge request
- [x] [:link:][step_embargo_end] **(IM)** ~~Inform original reporter (if external) that the security disclosure process is complete~~
- [x] [:link:][step_customers] **(Support)** Inform customers a fix has been released
### After Public Disclosure
- [x] [:link:][step_postmortem] **(First IM)** [Organize post-mortem meeting and make sure it happens](https://mattermost.isc.org/isc/pl/8bh5bx4betgbxx67zoxfa3zioc)
- [x] [:link:][step_tickets] **(Support)** Close support tickets
- [x] [:link:][step_regression] **(QA)** Merge a regression test reproducing the bug into all affected (and still maintained) branches
[step_deputy]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#pick-a-deputy-incident-manager
[step_respond]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#respond-to-the-bug-reporter
[step_etherpad]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#create-an-etherpad-for-post-mortem
[step_public_mrs]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#ensure-there-are-no-public-merge-requests-which-inadvertently-disclose-the-issue
[step_assign_cve_id]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#assign-a-cve-identifier
[step_note_cve_info]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#update-this-issue-with-the-assigned-cve-identifier-and-the-cvss-score
[step_versions_affected]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#determine-the-range-of-product-versions-affected-including-the-subscription-edition
[step_workarounds]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#determine-whether-workarounds-for-the-problem-exist
[step_coordinate]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#if-necessary-coordinate-with-other-parties
[step_earliest]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-and-send-out-earliest-notifications
[step_advisory_mr]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#create-a-merge-request-for-the-security-advisory-and-include-all-readily-available-information-in-it
[step_reproducer_mr]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-a-private-merge-request-containing-a-system-test-reproducing-the-problem
[step_notify_support]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#notify-support-when-a-reproducer-is-ready
[step_code_analysis]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-a-detailed-explanation-of-the-code-flow-triggering-the-problem
[step_fix_mr]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-a-private-merge-request-with-the-fix
[step_review_fix]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#ensure-the-merge-request-with-the-fix-is-reviewed-and-has-no-outstanding-discussions
[step_review_docs]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#review-the-documentation-changes-introduced-by-the-merge-request-with-the-fix
[step_backports]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-backports-of-the-merge-request-addressing-the-problem-for-all-affected-and-still-maintained-branches-of-a-given-product
[step_finish_advisory]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#finish-preparing-the-security-advisory
[step_meta_issue]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#create-or-update-the-private-issue-containing-links-to-fixes-reproducers-for-all-cves-fixed-in-a-given-release-cycle
[step_changes]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#bind-9-only-reserve-a-block-of-changes-placeholders-once-the-complete-set-of-vulnerabilities-fixed-in-a-given-release-cycle-is-determined
[step_merge_fixes]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#merge-the-cve-fixes-in-cve-identifier-order
[step_patches]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-a-standalone-patch-for-the-last-stable-release-of-each-affected-and-still-maintained-product-branch
[step_asn_releases]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-asn-releases-as-outlined-in-the-release-checklist
[step_send_asn]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#send-asn-to-eligible-customers
[step_preannouncement]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#bind-9-only-send-a-pre-announcement-email-to-the-bind-announce-mailing-list-to-alert-users-that-the-upcoming-release-will-include-security-fixes
[step_verify_asn]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#verify-that-all-asn-eligible-customers-have-received-the-notification-email
[step_check_customers]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#verify-that-any-new-or-reinstated-customers-have-received-the-notification-email
[step_packager_emails]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#send-notifications-to-os-packagers
[step_clearance]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#grant-support-clearance-to-proceed-with-public-release
[step_publish]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#publish-the-releases-as-outlined-in-the-release-checklist
[step_matrix]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#bind-9-only-update-vulnerability-matrix-in-the-knowledge-base
[step_publish_advisory]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#bump-document-version-for-the-security-advisory-and-publish-it-in-the-knowledge-base
[step_notifications]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#send-notification-emails-to-third-parties
[step_mitre]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#advise-mitre-about-the-disclosed-cves
[step_merge_advisory]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#merge-the-security-advisory-merge-request
[step_embargo_end]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#inform-original-reporter-if-external-that-the-security-disclosure-process-is-complete
[step_customers]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#inform-customers-a-fix-has-been-released
[step_postmortem]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#organize-post-mortem-meeting-and-make-sure-it-happens
[step_tickets]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#close-support-tickets
[step_regression]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#merge-a-regression-test-reproducing-the-bug-into-all-affected-and-still-maintained-branches
---
This crash is very easy to reproduce, at least on my Debian 11 virtual: `Linux BIND2 5.10.0-21-arm64 #1 SMP Debian 5.10.162-1 (2023-01-21) aarch64 GNU/Linux`
Create file called `dataset` with contents `rpzcname.myctl.com. A`
Execute `./queryperf -d dataset -l 300 -T 1000 -u 500 -s 127.0.0.1 -v -c` against BIND 9.18.15 with the following configuration:
```
controls {
inet 127.0.0.1 port 953 allow {
127.0.0.1/32;
} keys {
"rndc-key";
};
};
options {
directory "/usr/local/BIND/9.16.37/var/cache/bind";
pid-file "/usr/local/BIND/9.16.37/var/run/named.pid";
querylog no;
recursive-clients 500;
serial-query-rate 200;
tcp-clients 500;
transfers-per-ns 10;
version "unknown";
check-names slave ignore;
stale-answer-enable yes;
stale-answer-client-timeout 0;
stale-answer-ttl 31;
stale-cache-enable yes;
stale-refresh-time 90;
allow-transfer {
"localhost";
};
multi-master yes;
notify no;
};
statistics-channels {
inet 0.0.0.0 port 8080;
};
key "rndc-key" {
algorithm "hmac-sha256";
secret "????????????????????????????????????????????";
};
```
`named -V` output:
```
BIND 9.18.15 (Extended Support Version) <id:f53a076>
running on Linux aarch64 5.10.0-21-arm64 #1 SMP Debian 5.10.162-1 (2023-01-21)
built by make with '--enable-dnstap' '--with-libxml2' '--with-json-c' '--with-zlib' '--enable-full-report' '--prefix=/usr/local/BIND/9.18.15'
compiled by GCC 10.2.1 20210110
compiled with OpenSSL version: OpenSSL 1.1.1n 15 Mar 2022
linked to OpenSSL version: OpenSSL 1.1.1n 15 Mar 2022
compiled with libuv version: 1.40.0
linked to libuv version: 1.40.0
compiled with libnghttp2 version: 1.43.0
linked to libnghttp2 version: 1.43.0
compiled with libxml2 version: 2.9.10
linked to libxml2 version: 20910
compiled with json-c version: 0.15
linked to json-c version: 0.15
compiled with zlib version: 1.2.11
linked to zlib version: 1.2.11
compiled with protobuf-c version: 1.3.3
linked to protobuf-c version: 1.3.3
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): no
default paths:
named configuration: /usr/local/BIND/9.18.15/etc/named.conf
rndc configuration: /usr/local/BIND/9.18.15/etc/rndc.conf
DNSSEC root key: /usr/local/BIND/9.18.15/etc/bind.keys
nsupdate session key: /usr/local/BIND/9.18.15/var/run/named/session.key
named PID file: /usr/local/BIND/9.18.15/var/run/named/named.pid
named lock file: /usr/local/BIND/9.18.15/var/run/named/named.lock
```
See attached log debug log file from daemon start until crash in [debug.log](/uploads/33332e3d53c56074680133fef1439905/debug.log)
See attached stack trace in [gdb.txt](/uploads/378cd00895eaf320d043e89f3629a4d6/gdb.txt)
I also have a core file and could gather all of the details (binary / libraries etc...) if needed.
Other observation, once these log messages appear:
```
24-May-2023 18:06:16.022 rpz.myctl.com stale answer used, an attempt to refresh the RRset will still be made
```
BIND can only be stopped with a `kill -9`
also gave it a try with 9.16.37 on the same virtual. Same result of segfault.June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Tom KrizekTom Krizekhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4080named won't shut down after a timed out attempt to forward dynamic update2023-05-23T15:41:48ZTom Krizeknamed won't shut down after a timed out attempt to forward dynamic updateWhen `named` attempts to forward a dynamic DNS update to a non-responsive primary, it won't properly shutdown on `SIGTERM` once the TCP connection times out.
### Steps to reproduce
1. start named with this config
```
options {
port 53...When `named` attempts to forward a dynamic DNS update to a non-responsive primary, it won't properly shutdown on `SIGTERM` once the TCP connection times out.
### Steps to reproduce
1. start named with this config
```
options {
port 5353;
listen-on { 10.53.0.3; };
};
zone "noprimary" {
type secondary;
allow-update-forwarding { any; };
primaries port 5555 { 10.53.0.4; };
};
```
2. simulate a non-responsive primary - the key is to accept the TCP connection but simply don't respond to anything: `netcat -l -p 5555`
3. send the following nsupdate
```
local 10.53.0.1
server 10.53.0.3 5353
zone noprimary
update add unsigned.noprimary. 600 A 10.10.10.1
send
```
4. wait 30 seconds for the following log line to appear in named: `zone noprimary/IN: could not forward dynamic update to 10.53.0.4#5555: timed out`
5. send `SIGTERM` to namedhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4079session key is attached to multiple keyrings2023-05-26T12:29:25ZEvan Huntsession key is attached to multiple keyringsA TSIG key (`dns_tsigkey`) object can only be associated with one keyring at a time, but the session key was incorrectly being added to the keyrings for each view as they were configured.A TSIG key (`dns_tsigkey`) object can only be associated with one keyring at a time, but the session key was incorrectly being added to the keyrings for each view as they were configured.June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Evan HuntEvan Hunthttps://gitlab.isc.org/isc-projects/bind9/-/issues/4074Problem with stale-answer-enable true and clients-per-query increased2023-06-14T09:30:10ZDarren AnkneyProblem with stale-answer-enable true and clients-per-query increased### Summary
If `stale-answer-enable true;` is set and `clients-per-query 0;` is NOT set, and a large amount of queries per second for the same name are sent, then messages like `resolver: notice: clients-per-query increased to 15` are n...### Summary
If `stale-answer-enable true;` is set and `clients-per-query 0;` is NOT set, and a large amount of queries per second for the same name are sent, then messages like `resolver: notice: clients-per-query increased to 15` are not logged. The `clients-per-query` are supposed to be increased up to `max-clients-per-query`. It is currently unknown if this is a logging problem only or if the limit is not actually being incremented.
### BIND version used
```
BIND 9.16.37 (Extended Support Version) <id:2b2afb2>
running on Linux aarch64 5.10.0-21-arm64 #1 SMP Debian 5.10.162-1 (2023-01-21)
built by make with '--enable-dnstap' '--with-libxml2' '--with-json-c' '--with-zlib' '--enable-full-report' '--prefix=/usr/local/BIND/9.16.37'
compiled by GCC 10.2.1 20210110
compiled with OpenSSL version: OpenSSL 1.1.1n 15 Mar 2022
linked to OpenSSL version: OpenSSL 1.1.1n 15 Mar 2022
compiled with libuv version: 1.40.0
linked to libuv version: 1.40.0
compiled with libxml2 version: 2.9.10
linked to libxml2 version: 20910
compiled with json-c version: 0.15
linked to json-c version: 0.15
compiled with zlib version: 1.2.11
linked to zlib version: 1.2.11
compiled with protobuf-c version: 1.3.3
linked to protobuf-c version: 1.3.3
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): no
default paths:
named configuration: /usr/local/BIND/9.16.37/etc/named.conf
rndc configuration: /usr/local/BIND/9.16.37/etc/rndc.conf
DNSSEC root key: /usr/local/BIND/9.16.37/etc/bind.keys
nsupdate session key: /usr/local/BIND/9.16.37/var/run/named/session.key
named PID file: /usr/local/BIND/9.16.37/var/run/named/named.pid
named lock file: /usr/local/BIND/9.16.37/var/run/named/named.lock
```
### Steps to reproduce
In config file:
* set `stale-answer-enable true;`, `stale-answer-client-timeout 1800;`, `stale-answer-ttl 31;` and `stale-refresh-time 90;`
* Make sure `clients-per-query` and `max-clients-per-query` are not present in the config file
Run queryperf with a dataset that contains `www.microsoft.com A` like so:
`./queryperf -d dataset2 -l 300 -T 1000 -u 500 -s 192.168.115.227 -v -c`
where dataset2 contains: `www.microsoft.com A`
### What is the current *bug* behavior?
Messages such as `resolver: notice: clients-per-query increased to 15` are not logged.
### What is the expected *correct* behavior?
These messages should be logged as `clients-per-query` is increased toward `max-clients-per-query`.
Please note that merely changing `stale-answer-enable true;` to `stale-answer-enable false;` causes the messages to appear as shown:
```
17-May-2023 17:48:05.891 clients-per-query increased to 15
17-May-2023 17:48:06.163 clients-per-query increased to 20
17-May-2023 17:48:06.735 clients-per-query increased to 25
17-May-2023 17:48:06.903 clients-per-query increased to 30
```
### Relevant configuration files
```
options {
directory "/usr/local/BIND/9.16.37/var/cache/bind";
pid-file "/usr/local/BIND/9.16.37/var/run/named.pid";
querylog no;
recursive-clients 10000;
serial-query-rate 200;
tcp-clients 500;
transfers-per-ns 10;
version "unknown";
check-names slave ignore;
stale-answer-enable true;
stale-answer-client-timeout 1800;
stale-answer-ttl 31;
stale-refresh-time 90;
allow-transfer {
"localhost";
};
multi-master yes;
notify no;
};
statistics-channels {
inet 0.0.0.0 port 8080;
};
```
[RT22057](https://support.isc.org/Ticket/Display.html?id=22057)June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4072TCP dispatch timeout can cause shutdown hang2023-05-26T12:32:51ZEvan HuntTCP dispatch timeout can cause shutdown hangWhen a TCP dispatch times out, `tcp_recv()` cancels the oldest active query. If there are still other active queries waiting, then it resumes reading, but it doesn't reset the timer, so the subsequent queries can never time out. If the s...When a TCP dispatch times out, `tcp_recv()` cancels the oldest active query. If there are still other active queries waiting, then it resumes reading, but it doesn't reset the timer, so the subsequent queries can never time out. If the server shuts down while they're still pending then it can hang due to an un-detached network handle.
This can be observed in `main` by adding `sleep 15` immediately after calling `wait`:
```
echo_i "waiting for nsupdate to finish ($n)"
wait
n=`expr $n + 1`
echo_i "sleeping 15 seconds"
sleep 15
```June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)https://gitlab.isc.org/isc-projects/bind9/-/issues/4071named stops responding if an HTTP/2 listener is disabled whilst running2023-06-22T15:55:16ZGreg Choulesnamed stops responding if an HTTP/2 listener is disabled whilst running<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confident...<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confidential!
-->
### Summary
### BIND version used
BIND 9.18.14 (Extended Support Version) <id:2c5e22f>
running on Darwin x86_64 22.4.0 Darwin Kernel Version 22.4.0: Mon Mar 6 21:00:17 PST 2023; root:xnu-8796.101.5~3/RELEASE_X86_64
built by make with '--prefix=/usr/local/Cellar/bind/9.18.14' '--sysconfdir=/usr/local/etc/bind' '--localstatedir=/usr/local/var' '--with-json-c' '--with-libidn2=/usr/local/opt/libidn2' '--with-openssl=/usr/local/opt/openssl@3' '--without-lmdb' 'CC=clang' 'PKG_CONFIG_PATH=/usr/local/opt/json-c/lib/pkgconfig:/usr/local/opt/libidn2/lib/pkgconfig:/usr/local/opt/libnghttp2/lib/pkgconfig:/usr/local/opt/libuv/lib/pkgconfig:/usr/local/opt/openssl@3/lib/pkgconfig' 'PKG_CONFIG_LIBDIR=/usr/lib/pkgconfig:/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/13'
compiled by CLANG Apple LLVM 14.0.3 (clang-1403.0.22.14.1)
compiled with OpenSSL version: OpenSSL 3.1.0 14 Mar 2023
linked to OpenSSL version: OpenSSL 3.1.0 14 Mar 2023
compiled with libuv version: 1.44.2
linked to libuv version: 1.44.2
compiled with libnghttp2 version: 1.52.0
linked to libnghttp2 version: 1.52.0
compiled with libxml2 version: 2.9.13
linked to libxml2 version: 20913
compiled with json-c version: 0.16
linked to json-c version: 0.16
compiled with zlib version: 1.2.11
linked to zlib version: 1.2.11
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): yes
default paths:
named configuration: /usr/local/etc/bind/named.conf
rndc configuration: /usr/local/etc/bind/rndc.conf
DNSSEC root key: /usr/local/etc/bind/bind.keys
nsupdate session key: /usr/local/var/run/named/session.key
named PID file: /usr/local/var/run/named/named.pid
named lock file: /usr/local/var/run/named/named.lock
### Steps to reproduce
Start `named` with this:
```
tls tlsconf {
key-file "/usr/local/var/named/key.pem";
cert-file "/usr/local/var/named/cert.pem";
listen-on port 8888 tls tlsconf http httpsconf { 127.0.0.1; };
```
Comment the `listen-on` line then do `rndc reconfig`. `named` stops responding, both to queries and on the command channel
### What is the current *bug* behavior?
`named` stops responding to queries or `rndc` commands.
### What is the expected *correct* behavior?
`named` should still respond to commands and non-TLS queries.
### Relevant configuration files
```
controls {
inet 127.0.0.2 port 953 allow {
127.0.0.0/24;
} keys {
"rndc-key";
};
};
http "httpsconf" {
};
options {
bindkeys-file "bind.keys";
directory "/usr/local/var/named";
listen-on {
"any";
};
listen-on port 8888 tls "tlsconf" http "httpsconf" {
"any";
};
dnssec-accept-expired no;
dnssec-validation yes;
check-dup-records warn;
dnssec-dnskey-kskonly no;
dnssec-loadkeys-interval 60;
key-directory "keys";
update-check-ksk yes;
};
tls "tlsconf" {
key-file "/usr/local/var/named/key.pem";
cert-file "/usr/local/var/named/cert.pem";
prefer-server-ciphers no;
session-tickets no;
};
key "rndc-key" {
algorithm "hmac-sha256";
secret "????????????????????????????????????????????";
};
zone "tma.com" in {
type master;
file "db.tma.com";
allow-update {
"any";
};
notify yes;
};
zone "0.0.127.in-addr.arpa" in {
type master;
file "db.127.0.0";
};
```
### Relevant logs and/or screenshots
Apart from the usual logs you get with a reconfig, this is all that appears to say that the TLS listener now isn't.
```
15-May-2023 21:20:54.721 network: info: no longer listening on 127.0.0.1#8888
```
### Possible fixes
Some experimentation shows this to be related to DoH. Both of the following changes/reconfig do not cause the server to hang:
Start with:
```
listen-on port 8888 tls "tlsconf" http "httpsconf" {
"any";
};
```
Remove the `http` option and reconfig:
```
listen-on port 8888 tls "tlsconf" {
"any";
};
```
The server is still responsive and logs report that the TLS context has been updated, rather than is no longer listening:
```
16-May-2023 08:33:43.130 network: info: updating TLS context on 127.0.0.1#8888
16-May-2023 08:33:43.130 network: info: updating TLS context on 127.0.0.2#8888
16-May-2023 08:33:43.130 network: info: updating TLS context on 192.168.1.236#8888
```
This change is also reversible. The `http` option can be re-added and a reconfig is successful, wth similar log messages.June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Artem BoldarievArtem Boldarievhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4066resolv.conf parsing eats lines if more than 3 nameservers set2023-05-17T22:53:09ZRobert Bridgeresolv.conf parsing eats lines if more than 3 nameservers set<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confident...<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confidential!
-->
### Summary
The resolv.conf parsing used in nslookup eats the lines of resolv.conf if there are more than 3 nameservers defined in resolv.conf. This means that if there is an even number of nameservers defined, the first line following the nameservers gets silently eaten and ignored.
### BIND version used
Identified on CentOS 9 stream, confirmed from git on gentoo:
```
BIND 9.19.14-dev (Development Release) <id:562697e>
running on Linux x86_64 6.2.9-gentoo #2 SMP PREEMPT_DYNAMIC Mon May 1 09:27:12 BST 2023
built by make with default
compiled by GCC 13.1.0
compiled with OpenSSL version: OpenSSL 3.0.8 7 Feb 2023
linked to OpenSSL version: OpenSSL 3.0.8 7 Feb 2023
compiled with libuv version: 1.44.2
linked to libuv version: 1.44.2
compiled with liburcu version: 0.14.0
compiled with libnghttp2 version: 1.52.0
linked to libnghttp2 version: 1.52.0
compiled with libxml2 version: 2.11.3
linked to libxml2 version: 21103
compiled with json-c version: 0.16
linked to json-c version: 0.16
compiled with zlib version: 1.2.13
linked to zlib version: 1.2.13
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): no
TKEY mode 3 support (GSS-API): no
default paths:
named configuration: /usr/local/etc/named.conf
rndc configuration: /usr/local/etc/rndc.conf
nsupdate session key: /usr/local/var/run/named/session.key
named PID file: /usr/local/var/run/named/named.pid
named lock file: /usr/local/var/run/named/named.lock
```
### Steps to reproduce
Create resolv.conf with 4/6/8 nameserver entries, and a search line immediately after the last nameserver entry, e.g.:
```
nameserver 8.8.8.8
nameserver 8.8.8.8
nameserver 8.8.8.8
nameserver 8.8.8.8
search google.com
```
Run nslookup for a name that relies on the search line.
```
nslookup www
```
### What is the current *bug* behavior?
nslookup returns NXDOMAIN (typically)
```
# nslookup www
Server: 8.8.8.8
Address: 8.8.8.8#53
** server can't find www: NXDOMAIN
```
### What is the expected *correct* behavior?
nslookup should search the domains and find the relevant record
```
$ ./bin/dig/nslookup www
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: www.google.com
Address: 172.217.16.228
Name: www.google.com
Address: 2a00:1450:4009:820::2004
```
### Relevant configuration files
/etc/resolv.conf posted above
### Relevant logs and/or screenshots
### Possible fixesJune 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/4062CID 454693: A value assigned to addrinfo variable is never used in copy_nameh...2023-05-16T12:57:42ZMichal NowakCID 454693: A value assigned to addrinfo variable is never used in copy_namehook_lists()```
/lib/dns/adb.c: 2229 in copy_namehook_lists()
2223
2224 /*
2225 * Found a valid entry. Add it to the find's list.
2226 */
2227 inc_entry_refcnt(adb, entry, false);
2228 ISC_LIST_APPEND(find-...```
/lib/dns/adb.c: 2229 in copy_namehook_lists()
2223
2224 /*
2225 * Found a valid entry. Add it to the find's list.
2226 */
2227 inc_entry_refcnt(adb, entry, false);
2228 ISC_LIST_APPEND(find->list, addrinfo, publink);
>>> CID 454693: (UNUSED_VALUE)
>>> Assigning value "NULL" to "addrinfo" here, but that stored value is overwritten before it can be used.
2229 addrinfo = NULL;
2230 nextv4:
2231 UNLOCK(&adb->entrylocks[bucket]);
2232 bucket = DNS_ADB_INVALIDBUCKET;
2233 namehook = ISC_LIST_NEXT(namehook, plink);
2234 }
/lib/dns/adb.c: 2264 in copy_namehook_lists()
2258
2259 /*
2260 * Found a valid entry. Add it to the find's list.
2261 */
2262 inc_entry_refcnt(adb, entry, false);
2263 ISC_LIST_APPEND(find->list, addrinfo, publink);
>>> CID 454693: (UNUSED_VALUE)
>>> Assigning value "NULL" to "addrinfo" here, but that stored value is overwritten before it can be used.
2264 addrinfo = NULL;
2265 nextv6:
2266 UNLOCK(&adb->entrylocks[bucket]);
2267 bucket = DNS_ADB_INVALIDBUCKET;
2268 namehook = ISC_LIST_NEXT(namehook, plink);
2269 }
```
This was identified by the new Coverity Scan 2022.12 version, so probably not a new issue, on ~"v9.16" and ~"v9.18".https://gitlab.isc.org/isc-projects/bind9/-/issues/4059Oracle Linux 8 shell doesn't always restore environment variable correctly2023-08-02T08:49:35ZMark AndrewsOracle Linux 8 shell doesn't always restore environment variable correctlyJob [#3374668](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3374668) failed for 0592aaa4a97351f736418b2bbc53e89113a578c5:
I saw that dig was making AAAA queries instead of A queries by default a couple of times developing !7888. Ea...Job [#3374668](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3374668) failed for 0592aaa4a97351f736418b2bbc53e89113a578c5:
I saw that dig was making AAAA queries instead of A queries by default a couple of times developing !7888. Earlier in the resolver test we use `HOME="$(pwd)" dig_with_opts @10.53.0.4 . > dig.out.1.${n} || ret=1` a few times. This shouldn't have a long term effect on HOME but that is the only explanation that makes sense. Creating an explicit sub shell should fix the issue. For !7888 I explicitly set the type.
Note below the type was not specified but should default to A.
```
% more dig.out.60
; <<>> DiG 9.19.14-dev <<>> -p 14345 +tcp @10.53.0.5 options-formerr
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54036
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 31786a72177b15f101000000645c36ee6cd499386040212e (good)
;; QUESTION SECTION:
;options-formerr. IN AAAA
;; AUTHORITY SECTION:
options-formerr. 1 IN SOA . . 0 0 0 0 0
;; Query time: 0 msec
;; SERVER: 10.53.0.5#14345(10.53.0.5) (TCP)
;; WHEN: Thu May 11 00:29:34 UTC 2023
;; MSG SIZE rcvd: 106
%
```August 2023 (9.16.43, 9.16.43-S1, 9.18.18, 9.18.18-S1, 9.19.16)https://gitlab.isc.org/isc-projects/bind9/-/issues/4055[CVE-2023-2828] named's configured cache size limit can be significantly exce...2023-11-15T08:47:01ZShoham Danino[CVE-2023-2828] named's configured cache size limit can be significantly exceeded| Quick Links | :link: |
| ------------------------ | ------------------------------------------------------------------------------|
| Incident Manager:...| Quick Links | :link: |
| ------------------------ | ------------------------------------------------------------------------------|
| Incident Manager: | @michal |
| Deputy Incident Manager: | @aram |
| Public Disclosure Date: | 2023-06-21 |
| CVSS Score: | [7.5][cvss_score] |
| Security Advisory: | isc-private/printing-press!54 |
| Mattermost Channel: | [CVE-2023-2828: max-cache-size can be significantly exceeded][mattermost_url] |
| Support Ticket: | N/A |
| Release Checklist: | https://gitlab.isc.org/isc-projects/bind9/-/issues/4123 |
| Post-mortem Etherpad: | [postmortem-2023-06][postmortem_url] |
[cvss_score]: https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H&version=3.1
[mattermost_url]: https://mattermost.isc.org/isc/channels/cve-2023-2828-max-cache-size
[postmortem_url]: https://pad.isc.org/p/postmortem-2023-06
:bulb: **Click [here][checklist_explanations] (internal resource) for general information about the security incident handling process.**
[checklist_explanations]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations
### Earlier Than T-5
- [x] [:link:][step_deputy] **(IM)** Pick a Deputy Incident Manager
- [x] [:link:][step_respond] **(IM)** [Respond to the bug reporter](#note_373642)
- [x] [:link:][step_etherpad] **(IM)** [Create an Etherpad for post-mortem][postmortem_url]
- [x] [:link:][step_public_mrs] **(SwEng)** Ensure there are no public merge requests which inadvertently disclose the issue
- [x] [:link:][step_assign_cve_id] **(IM)** [Assign a CVE identifier](#note_374026)
- [x] [:link:][step_note_cve_info] **(SwEng)** Update this issue with the assigned CVE identifier and the CVSS score
- [x] [:link:][step_versions_affected] **(SwEng)** [Determine the range of product versions affected (including the Subscription Edition)](#note_374064)
- [x] [:link:][step_workarounds] **(SwEng)** [Determine whether workarounds for the problem exist](#note_374750)
- [x] [:link:][step_coordinate] **(SwEng)** ~~If necessary, coordinate with other parties~~
- [x] [:link:][step_earliest] **(Support)** Prepare and send out "earliest" notifications
- [x] [:link:][step_advisory_mr] **(Support)** [Create a merge request for the Security Advisory and include all readily available information in it](isc-private/printing-press!54)
- [x] [:link:][step_reproducer_mr] **(SwEng)** [Prepare a private merge request containing a system test reproducing the problem](isc-private/bind9!519)
- [x] [:link:][step_notify_support] **(SwEng)** [Notify Support when a reproducer is ready](https://mattermost.isc.org/isc/pl/bfjbhijg1jnqzxoix8q6uhsp7e)
- [x] [:link:][step_code_analysis] **(SwEng)** [Prepare a detailed explanation of the code flow triggering the problem](#note_372558)
- [x] [:link:][step_fix_mr] **(SwEng)** [Prepare a private merge request with the fix](isc-private/bind9!520)
- [x] [:link:][step_review_fix] **(SwEng)** [Ensure the merge request with the fix is reviewed and has no outstanding discussions](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/520#note_378278)
- [x] [:link:][step_review_docs] **(Support)** [Review the documentation changes introduced by the merge request with the fix](https://mattermost.isc.org/isc/pl/ys9ngz8hpffitrdefochzi1ayh)
- [x] [:link:][step_backports] **(SwEng)** Prepare backports of the merge request addressing the problem for all affected ([and](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/527) [still](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/528) [maintained](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/529)) branches of a given product
- [x] [:link:][step_finish_advisory] **(Support)** [Finish preparing the Security Advisory](https://mattermost.isc.org/isc/pl/nt6beetinpnwzg8678sb5fi79r)
- [x] [:link:][step_meta_issue] **(QA)** [Create (or update) the private issue containing links to fixes & reproducers for all CVEs fixed in a given release cycle](https://gitlab.isc.org/isc-private/bind9/-/issues/68)
- [x] [:link:][step_changes] **(QA)** (BIND 9 only) [Reserve a block of `CHANGES` placeholders once the complete set of vulnerabilities fixed in a given release cycle is determined](isc-projects/bind9!8010)
- [x] [:link:][step_merge_fixes] **(QA)** Merge the CVE fixes in CVE identifier order
- [x] [:link:][step_patches] **(QA)** [Prepare a standalone patch for the last stable release of each affected (and still maintained) product branch](https://mattermost.isc.org/isc/pl/apq5r8ir7ffnb8br3bdpkqbg5a)
- [x] [:link:][step_asn_releases] **(QA)** [Prepare ASN releases (as outlined in the Release Checklist)](https://mattermost.isc.org/isc/pl/gbe5fz3bypfixqt67b85tapruc)
### At T-5
- [x] [:link:][step_send_asn] **(Support)** [Send ASN to eligible customers](https://mattermost.isc.org/isc/pl/43d9p7bou7g79e1zbnp91fooxr)
- [x] [:link:][step_preannouncement] **(Support)** [(BIND 9 only) Send a pre-announcement email to the *bind-announce* mailing list to alert users that the upcoming release will include security fixes](isc-private/printing-press!57)
### At T-4
- [x] [:link:][step_verify_asn] **(Support)** Verify that all ASN-eligible customers have received the notification email
### At T-1
- [x] [:link:][step_check_customers] **(Support)** Verify that any new or reinstated customers have received the notification email
- [x] [:link:][step_packager_emails] **(First IM)** [Send notifications to OS packagers](https://gitlab.isc.org/isc-private/printing-press/-/merge_requests/58#note_382746)
### On the Day of Public Disclosure
- [x] [:link:][step_clearance] **(IM)** [Grant Support clearance to proceed with public release](https://mattermost.isc.org/isc/pl/kx7cyamrwiysdf3wqgq7qwo4me)
- [x] [:link:][step_publish] **(Support)** Publish the releases (as outlined in the release checklist)
- [x] [:link:][step_matrix] **(Support)** (BIND 9 only) Update vulnerability matrix in the Knowledge Base
- [x] [:link:][step_publish_advisory] **(Support)** Bump Document Version for the Security Advisory and publish it in the Knowledge Base
- [x] [:link:][step_notifications] **(First IM)** [Send notification emails to third parties](https://gitlab.isc.org/isc-private/printing-press/-/merge_requests/61#note_383415)
- [x] [:link:][step_mitre] **(First IM)** [Advise MITRE about the disclosed CVEs](https://gitlab.isc.org/isc-private/printing-press/-/merge_requests/54#note_383433)
- [x] [:link:][step_merge_advisory] **(First IM)** Merge the Security Advisory merge request
- [x] [:link:][step_embargo_end] **(IM)** [Inform original reporter (if external) that the security disclosure process is complete](#note_383448)
- [x] [:link:][step_customers] **(Support)** Inform customers a fix has been released
### After Public Disclosure
- [x] [:link:][step_postmortem] **(First IM)** [Organize post-mortem meeting and make sure it happens](https://mattermost.isc.org/isc/pl/8bh5bx4betgbxx67zoxfa3zioc)
- [x] [:link:][step_tickets] **(Support)** Close support tickets
- [x] [:link:][step_regression] **(QA)** Merge a regression test reproducing the bug into all affected (and still maintained) branches
[step_deputy]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#pick-a-deputy-incident-manager
[step_respond]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#respond-to-the-bug-reporter
[step_etherpad]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#create-an-etherpad-for-post-mortem
[step_public_mrs]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#ensure-there-are-no-public-merge-requests-which-inadvertently-disclose-the-issue
[step_assign_cve_id]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#assign-a-cve-identifier
[step_note_cve_info]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#update-this-issue-with-the-assigned-cve-identifier-and-the-cvss-score
[step_versions_affected]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#determine-the-range-of-product-versions-affected-including-the-subscription-edition
[step_workarounds]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#determine-whether-workarounds-for-the-problem-exist
[step_coordinate]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#if-necessary-coordinate-with-other-parties
[step_earliest]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-and-send-out-earliest-notifications
[step_advisory_mr]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#create-a-merge-request-for-the-security-advisory-and-include-all-readily-available-information-in-it
[step_reproducer_mr]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-a-private-merge-request-containing-a-system-test-reproducing-the-problem
[step_notify_support]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#notify-support-when-a-reproducer-is-ready
[step_code_analysis]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-a-detailed-explanation-of-the-code-flow-triggering-the-problem
[step_fix_mr]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-a-private-merge-request-with-the-fix
[step_review_fix]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#ensure-the-merge-request-with-the-fix-is-reviewed-and-has-no-outstanding-discussions
[step_review_docs]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#review-the-documentation-changes-introduced-by-the-merge-request-with-the-fix
[step_backports]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-backports-of-the-merge-request-addressing-the-problem-for-all-affected-and-still-maintained-branches-of-a-given-product
[step_finish_advisory]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#finish-preparing-the-security-advisory
[step_meta_issue]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#create-or-update-the-private-issue-containing-links-to-fixes-reproducers-for-all-cves-fixed-in-a-given-release-cycle
[step_changes]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#bind-9-only-reserve-a-block-of-changes-placeholders-once-the-complete-set-of-vulnerabilities-fixed-in-a-given-release-cycle-is-determined
[step_merge_fixes]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#merge-the-cve-fixes-in-cve-identifier-order
[step_patches]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-a-standalone-patch-for-the-last-stable-release-of-each-affected-and-still-maintained-product-branch
[step_asn_releases]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#prepare-asn-releases-as-outlined-in-the-release-checklist
[step_send_asn]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#send-asn-to-eligible-customers
[step_preannouncement]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#bind-9-only-send-a-pre-announcement-email-to-the-bind-announce-mailing-list-to-alert-users-that-the-upcoming-release-will-include-security-fixes
[step_verify_asn]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#verify-that-all-asn-eligible-customers-have-received-the-notification-email
[step_check_customers]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#verify-that-any-new-or-reinstated-customers-have-received-the-notification-email
[step_packager_emails]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#send-notifications-to-os-packagers
[step_clearance]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#grant-support-clearance-to-proceed-with-public-release
[step_publish]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#publish-the-releases-as-outlined-in-the-release-checklist
[step_matrix]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#bind-9-only-update-vulnerability-matrix-in-the-knowledge-base
[step_publish_advisory]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#bump-document-version-for-the-security-advisory-and-publish-it-in-the-knowledge-base
[step_notifications]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#send-notification-emails-to-third-parties
[step_mitre]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#advise-mitre-about-the-disclosed-cves
[step_merge_advisory]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#merge-the-security-advisory-merge-request
[step_embargo_end]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#inform-original-reporter-if-external-that-the-security-disclosure-process-is-complete
[step_customers]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#inform-customers-a-fix-has-been-released
[step_postmortem]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#organize-post-mortem-meeting-and-make-sure-it-happens
[step_tickets]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#close-support-tickets
[step_regression]: https://gitlab.isc.org/isc-private/isc-wiki/-/wikis/Security-Incident-Handling-Checklist-Explanations#merge-a-regression-test-reproducing-the-bug-into-all-affected-and-still-maintained-branches
---
### Summary
This vulnerability results in high memory cache usage for a DNS resolver, even larger than the maximum cache size configured. This happens when the resolver gets around 20,000 requests in several minutes or hours.
For example, with a 250 QPS rate, 1000MB RAM is used after 80 seconds when cache max size is configured to 32MB (the results example is attached to this message).
### BIND version used
BIND 9.16.40 (Extended Support Version) <id:113a865>
### Steps to reproduce
We reproduce NRDelegationAttack with some changes, for more details: https://www.usenix.org/system/files/sec23fall-prepub-309-afek.pdf
1. set the maximum cache size to 32MB:
in named.conf.option (attached example:[named.conf.options](/uploads/bd5dadb4674f120dbe96fd6ca5d060ba/named.conf.options)):
```
options {
...
max-cache-size 32m;
...
}
```
2. Run the resolver: `named -g -c /etc/named.conf`
3. Run psrecord for testing the RAM usage of the resolver: `psrecord NAMED_PID --interval 1 --plot OUTPUT_FILE.png`
4. Option a:
> Request my domain (shoham-shani.online) up to 50,000 dns queries (my authoritative ip address is 74.234.116.29):
`dig shoham{count}.shoham-shani.online. @resolver_ip (count is from 0 to 49,999)` (you can use dnsperf: `dnsperf -d queries.txt -s resolver_ip -v -Q 250`, an example to queries.txt is attached: [queries.txt](/uploads/3a710d8c8416c83ce0469dc7d6b7c92a/queries.txt)
> You can provide us with a test resolver that you want us to attack, and we will perform the attack from our client side at the time we will agree on.
5. option b:
> Create a zone file (example is attached) that has 1500 referrals per one request, you can use this script for that:
with open('zonfile.txt', 'w') as f:
for i in range(1, 50000):
for j in range(0, 1500):
print(f'shoham{i} 8600 IN NS attack{j}.auth{j}.shoham.store.',file=f)
[shoham-shani.online_zonefile_example.txt](/uploads/1cce5b5b5a7118205bde4f199e1c3404/shoham-shani.online_zonefile_example.txt)
> Create another zonefile that answers all shoham{i}.shoham.store. request:
For example: `* IN A 127.0.0.1`
[shoham.store_zonefile_example_copy.txt](/uploads/1fbda03a3a24fdffe95df466d3b031e6/shoham.store_zonefile_example_copy.txt)
> Request 50,000 dns queries as I described in option a.
6. Take a dump of the cache and examine its size: `rndc dumpdb -cache`
7. Stop the resolver service and download the OUTPUT_FILE.png image, examining RAM usage.
note: we checked the bug with 32MB, 64MB and 1GB max-cache-size and with rate of 1,5,10,13,100,250 QPS (all the results are attached)
For 1 QPS I got 440MB RAM used after 8,000 seconds for max-cache-size 32MB
![attack_1_qps](/uploads/4e918e5100db74bae11edf5df8a485bd/attack_1_qps.png)
For 5 QPS I got 840MB RAM used after 4,000 seconds for max-cache-size 32MB
![attack_5_qps](/uploads/c55932c784c402d868af2d306efb6795/attack_5_qps.png)
For 10 QPS I got 840MB RAM used after 2,000 seconds for max-cache-size 32MB
![attack_10_qps](/uploads/d4293561ddc6ba22d8ee203fb2244249/attack_10_qps.png)
For 100 QPS I got 840MB RAM used after 200 seconds for max-cache-size 32MB
![attack_100_qps](/uploads/a18150c0b47480e8ff9d5df5d68afce4/attack_100_qps.png)
For 250 QPS I got 1000MB RAM used after 80 seconds for max-cache-size 32MB
![attack_250_qps](/uploads/116feeb2217d76b84aa3427a0a52db32/attack_250_qps.png)
For 13 QPS I got 1550MB RAM used after 1150 seconds for max-cache-size 1000MB
![attack_1GB_cache](/uploads/bc7a472599212d5b1feed1563ee014bb/attack_1GB_cache.jpeg)
### What is the current *bug* behavior?
1. The cache size expands beyond the limit resulting in an increasing amount of memory being allocated. In addition, if there is no memory available on the machine, the resolver service will crash.
2. A free memory action is not performed for the referral list buffer, which results in an increase in memory allocation for buffers (dns_rdataslab_fromrdataset function in line 272 of the rdataslab.c file).
3. It seems the cache size calculation does not consider authoritative nameservers' refferal answers, although they are stored in the cache.
4. High and low watermarks are set incorrectly to 0, which means the resolver is unaware that the memory usage exceeds the maximum level and does not reduce it.
### What is the expected *correct* behavior?
1. The cache size should be the maximum size we configured.
2. There should be a free memory action to the buffer (rawbuf in rdataslab.c file)
3. In order to calculate cache size, it is necessary to take into account referral list and perform a deletion when the cache exceeds the configured limit.
### Relevant configuration files
configuration file:
> named.conf.options
zonefiles:
> shoham-shani.online_zonefile_example.txt
> shoham.store_zonefile_example copy.txt
### Relevant logs and/or screenshots
Tests are attached
### Possible fixes
1. Free the buffer:
```
free_rawbuf:
isc_mem_put(mctx, rawbuf, buflen);
```
please add the following to this issue:
Yehuda Afek, yehuda.afek@gmail.com /cc @afek
Anat Bremler-Barr, anat.bremlerbarr@gmail.com /cc @anat_bremler_barr
Yuval Shavitt, shavitt@eng.tau.ac.ilJune 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Tom KrizekTom Krizekhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4054Reference counting failure "REQUIRE(ptr != ((void *)0))" in db.c2023-05-17T13:16:28ZMichal NowakReference counting failure "REQUIRE(ptr != ((void *)0))" in db.cJob [#3368111](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3368111) failed for 9b28225611f34dac71bd3885e15ad554ccb60a97.
`ns3` of the `inline` test hit a reference counting issue `REQUIRE(ptr != ((void *)0))` in `db.c`.
```
D:/bui...Job [#3368111](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3368111) failed for 9b28225611f34dac71bd3885e15ad554ccb60a97.
`ns3` of the `inline` test hit a reference counting issue `REQUIRE(ptr != ((void *)0))` in `db.c`.
```
D:/builds/isc-projects/bind9/bin/tests/system/inline:Core was generated by `/builds/isc-projects/bind9/bin/named/.libs/named -D inline-ns3 -X named.lock -m'.
D:/builds/isc-projects/bind9/bin/tests/system/inline:Program terminated with signal SIGABRT, Aborted.
D:/builds/isc-projects/bind9/bin/tests/system/inline:#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
D:/builds/isc-projects/bind9/bin/tests/system/inline:[Current thread is 1 (Thread 0x7f6719e53680 (LWP 1823))]
D:/builds/isc-projects/bind9/bin/tests/system/inline:#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
D:/builds/isc-projects/bind9/bin/tests/system/inline:#1 0x00007f67205f5d2f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
D:/builds/isc-projects/bind9/bin/tests/system/inline:#2 0x00007f67205a6ef2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
D:/builds/isc-projects/bind9/bin/tests/system/inline:#3 0x00007f6720591472 in __GI_abort () at ./stdlib/abort.c:79
D:/builds/isc-projects/bind9/bin/tests/system/inline:#4 0x000055a4e5ebb96a in assertion_failed (file=<optimized out>, line=150, type=isc_assertiontype_require, cond=0x7f67211a7b8d "ptr != ((void *)0)") at main.c:225
D:/builds/isc-projects/bind9/bin/tests/system/inline:#5 0x00007f672121f29a in isc_assertion_failed (file=file@entry=0x7f672118d510 "db.c", line=line@entry=150, type=type@entry=isc_assertiontype_require, cond=cond@entry=0x7f67211a7b8d "ptr != ((void *)0)") at assertions.c:48
D:/builds/isc-projects/bind9/bin/tests/system/inline:#6 0x00007f6721011801 in dns_db_ref (ptr=<optimized out>) at db.c:150
D:/builds/isc-projects/bind9/bin/tests/system/inline:#7 0x00007f67210118ad in dns_db_attach (ptr=0x0, ptrp=ptrp@entry=0x7f6719e51f90) at db.c:150
D:/builds/isc-projects/bind9/bin/tests/system/inline:#8 0x00007f6721169559 in zone_resigninc (zone=zone@entry=0x55a4e64d2710) at zone.c:6828
D:/builds/isc-projects/bind9/bin/tests/system/inline:#9 0x00007f672117214f in zone_maintenance (zone=<optimized out>) at zone.c:11100
D:/builds/isc-projects/bind9/bin/tests/system/inline:#10 0x00007f67211750e6 in zone_timer (arg=<optimized out>) at zone.c:14634
D:/builds/isc-projects/bind9/bin/tests/system/inline:#11 0x00007f672124776e in timer_cb (handle=<optimized out>) at timer.c:111
D:/builds/isc-projects/bind9/bin/tests/system/inline:#12 0x00007f6720a35cbe in ?? () from /lib/x86_64-linux-gnu/libuv.so.1
D:/builds/isc-projects/bind9/bin/tests/system/inline:#13 0x00007f6720a3999f in uv_run () from /lib/x86_64-linux-gnu/libuv.so.1
D:/builds/isc-projects/bind9/bin/tests/system/inline:#14 0x00007f6721234748 in loop_thread (arg=arg@entry=0x55a4e613b4b8) at loop.c:311
D:/builds/isc-projects/bind9/bin/tests/system/inline:#15 0x00007f6721246012 in thread_body (wrap=0x55a4e61782f0) at thread.c:89
D:/builds/isc-projects/bind9/bin/tests/system/inline:#16 thread_run (wrap=0x55a4e61782f0) at thread.c:104
D:/builds/isc-projects/bind9/bin/tests/system/inline:#17 0x00007f67205f3fd4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
D:/builds/isc-projects/bind9/bin/tests/system/inline:#18 0x00007f6720673820 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100
```
It seems that `named` crashed shortly before the `testing that inline signing works with inactive ZSK and active KSK` test.
CI artifacts were saved in the job.June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4049Detect FORMERR with an echoed DNS COOKIE client cookie and retry without DNS ...2023-07-05T10:39:40ZMark AndrewsDetect FORMERR with an echoed DNS COOKIE client cookie and retry without DNS COOKIEServers that send this sort of response to request with DNS COOKIE are broken as they don't comply with RFC 6891 but they are still at around .5% of servers and are declining.Servers that send this sort of response to request with DNS COOKIE are broken as they don't comply with RFC 6891 but they are still at around .5% of servers and are declining.June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)https://gitlab.isc.org/isc-projects/bind9/-/issues/4047Assertion failure in dns_resolver_attach() at resolver.c:105992023-06-29T20:15:07ZMichal NowakAssertion failure in dns_resolver_attach() at resolver.c:10599Job [#3359953](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3359953) failed for 5a4ea1236bbef522ef13f5189a07de52e980eb65.
```
resolver.c:10599: REQUIRE(!__extension__ ({ __auto_type __atomic_load_ptr = ((&source->exiting)); __typeof...Job [#3359953](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3359953) failed for 5a4ea1236bbef522ef13f5189a07de52e980eb65.
```
resolver.c:10599: REQUIRE(!__extension__ ({ __auto_type __atomic_load_ptr = ((&source->exiting)); __typeof__ ((void)0, *__atomic_load_ptr) __atomic_load_tmp; __atomic_load (__atomic_load_ptr, &__atomic_load_tmp, (memory_order_acquire)); __atomic_load_tmp; })) failed
```
```
I:shutdown:Core dump(s) found: /builds/isc-projects/bind9/bin/tests/system/shutdown/resolver/core.6454
D:shutdown:backtrace from /builds/isc-projects/bind9/bin/tests/system/shutdown/resolver/core.6454:
D:shutdown:--------------------------------------------------------------------------------
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:Core was generated by `/builds/isc-projects/bind9/bin/named/.libs/lt-named -c /builds/isc-projects/bin'.
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:Program terminated with signal SIGABRT, Aborted.
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#0 __restore_sigs (set=set@entry=0x7f4f2a0c0eb0) at ./arch/x86_64/syscall_arch.h:40
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:[Current thread is 1 (LWP 6525)]
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#0 __restore_sigs (set=set@entry=0x7f4f2a0c0eb0) at ./arch/x86_64/syscall_arch.h:40
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#1 0x00007f4f2d3ca382 in raise (sig=sig@entry=6) at src/signal/raise.c:11
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#2 0x00007f4f2d39dec5 in abort () at src/exit/abort.c:11
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#3 0x00005613ffb84eb8 in assertion_failed (file=<optimized out>, line=<optimized out>, type=<optimized out>, cond=0x7f4f2cda1b90 "!__extension__ ({ __auto_type __atomic_load_ptr = ((&source->exiting)); __typeof__ ((void)0, *__atomic_load_ptr) __atomic_load_tmp; __atomic_load (__atomic_load_ptr, &__atomic_load_tmp, (memory_order_"...) at main.c:239
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#4 0x00007f4f2ce23ae5 in isc_assertion_failed (file=file@entry=0x7f4f2cda2586 "resolver.c", line=line@entry=10599, type=type@entry=isc_assertiontype_require, cond=cond@entry=0x7f4f2cda1b90 "!__extension__ ({ __auto_type __atomic_load_ptr = ((&source->exiting)); __typeof__ ((void)0, *__atomic_load_ptr) __atomic_load_tmp; __atomic_load (__atomic_load_ptr, &__atomic_load_tmp, (memory_order_"...) at assertions.c:48
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#5 0x00007f4f2cce5529 in dns_resolver_attach (source=source@entry=0x7f4f2923a100, targetp=targetp@entry=0x7f4f2821d008) at resolver.c:10599
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#6 0x00007f4f2cce76ed in fctx_create (res=res@entry=0x7f4f2923a100, task=task@entry=0x7f4f28958d80, name=name@entry=0x7f4f2621a008, type=type@entry=28, domain=domain@entry=0x7f4f2a0c1f50, nameservers=nameservers@entry=0x7f4f2a0c1ee0, client=<optimized out>, options=<optimized out>, bucketnum=<optimized out>, depth=<optimized out>, qc=<optimized out>, fctxp=<optimized out>) at resolver.c:4778
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#7 0x00007f4f2ccebca6 in dns_resolver_createfetch (res=<optimized out>, name=name@entry=0x7f4f2621a008, type=type@entry=28, domain=domain@entry=0x7f4f2a0c1f50, nameservers=nameservers@entry=0x7f4f2a0c1ee0, forwarders=forwarders@entry=0x0, client=<optimized out>, id=<optimized out>, options=<optimized out>, depth=<optimized out>, qc=<optimized out>, task=<optimized out>, action=<optimized out>, arg=<optimized out>, rdataset=<optimized out>, sigrdataset=<optimized out>, fetchp=<optimized out>) at resolver.c:10927
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#8 0x00007f4f2cbf4213 in fetch_name (adbname=adbname@entry=0x7f4f2621a000, start_at_zone=start_at_zone@entry=true, depth=depth@entry=1, qc=qc@entry=0x7f4f28b26580, type=type@entry=28) at adb.c:4077
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#9 0x00007f4f2cbfc32e in dns_adb_createfind (adb=<optimized out>, task=0x7f4f28955cc0, action=action@entry=0x7f4f2cceda27 <fctx_finddone>, arg=arg@entry=0x7f4f28b1f800, name=name@entry=0x7f4f2a0c2c50, qname=0x7f4f28b1f810, qtype=<optimized out>, options=<optimized out>, now=<optimized out>, target=<optimized out>, port=<optimized out>, depth=<optimized out>, qc=<optimized out>, findp=<optimized out>) at adb.c:3148
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#10 0x00007f4f2ccea738 in findname (fctx=fctx@entry=0x7f4f28b1f800, name=name@entry=0x7f4f2a0c2c50, port=port@entry=0, options=<optimized out>, options@entry=15, flags=flags@entry=0, now=<optimized out>, overquota=<optimized out>, need_alternate=<optimized out>, no_addresses=<optimized out>) at resolver.c:3410
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#11 0x00007f4f2ccecd14 in fctx_getaddresses (badcache=<optimized out>, fctx=0x7f4f28b1f800) at resolver.c:3747
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#12 fctx_try (fctx=<optimized out>, fctx@entry=0x7f4f28b1f800, retrying=retrying@entry=true, badcache=badcache@entry=false) at resolver.c:4144
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#13 0x00007f4f2ccf4c48 in resume_qmin (task=<optimized out>, event=<optimized out>) at resolver.c:4390
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#14 0x00007f4f2ce44071 in task_run (task=0x7f4f28955cc0) at task.c:815
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#15 isc_task_run (task=0x7f4f28955cc0) at task.c:896
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#16 0x00007f4f2ce08be5 in isc__nm_async_task (worker=worker@entry=0x7f4f2ba39cc0, ev0=ev0@entry=0x7f4f2b44a980) at netmgr/netmgr.c:848
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#17 0x00007f4f2ce1082d in process_netievent (worker=worker@entry=0x7f4f2ba39cc0, ievent=0x7f4f2b44a980) at netmgr/netmgr.c:920
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#18 0x00007f4f2ce1121b in process_queue (worker=worker@entry=0x7f4f2ba39cc0, type=type@entry=NETIEVENT_TASK) at netmgr/netmgr.c:1013
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#19 0x00007f4f2ce11cd2 in process_all_queues (worker=0x7f4f2ba39cc0) at netmgr/netmgr.c:767
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#20 async_cb (handle=0x7f4f2ba3a020) at netmgr/netmgr.c:796
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#21 0x00007f4f2c707b4a in ?? () from /usr/lib/libuv.so.1
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#22 0x00007f4f2c7174c9 in uv.io_poll () from /usr/lib/libuv.so.1
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#23 0x00007f4f2c70818b in uv_run () from /usr/lib/libuv.so.1
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#24 0x00007f4f2ce114ea in nm_thread (worker0=0x7f4f2ba39cc0) at netmgr/netmgr.c:698
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#25 0x00007f4f2ce4d044 in isc__trampoline_run (arg=0x7f4f2b5f27c0) at trampoline.c:189
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#26 0x00007f4f2d3d708b in start (p=0x7f4f2a0c6f48) at src/thread/pthread_create.c:203
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:#27 0x00007f4f2d3d938e in __clone () at src/thread/x86_64/clone.s:22
D:/builds/isc-projects/bind9/bin/tests/system/shutdown:Backtrace stopped: frame did not save the PC
```
Artifacts were saved as part of the CI job.July 2023 (9.18.17, 9.18.17-S1, 9.19.15)Mark AndrewsMark Andrewshttps://gitlab.isc.org/isc-projects/bind9/-/issues/4046[ISC-support #22037] rndc times out in 30 seconds when using BIND 9.18.112023-05-17T13:23:06ZEverett Fulton[ISC-support #22037] rndc times out in 30 seconds when using BIND 9.18.11A Support customer is reporting that 'rndc' is timing out in 30 seconds, where the timeout was previously 60 seconds. Their complex configuration and multiple catalog zones creates a requirement for more time for a reconfig.
https://su...A Support customer is reporting that 'rndc' is timing out in 30 seconds, where the timeout was previously 60 seconds. Their complex configuration and multiple catalog zones creates a requirement for more time for a reconfig.
https://support.isc.org/Ticket/Display.html?id=22037
As mentioned in the BIND/Support meeting on 3 May, it may be possible to backport the 60s timeout from 9.19.June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Evan HuntEvan Hunthttps://gitlab.isc.org/isc-projects/bind9/-/issues/4044nslookup reports timeout if input lookup is delayed2023-09-09T18:13:01ZPetr Menšíknslookup reports timeout if input lookup is delayed<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confident...<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confidential!
-->
### Summary
nslookup reports timeout if input lookup is delayed
### BIND version used
```
BIND 9.18.14 (Extended Support Version) <id:>
running on Linux x86_64 6.2.12-300.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Apr 20 23:05:25 UTC 2023
built by make with '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--runstatedir=/run' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--localstatedir=/var' '--with-pic' '--disable-static' '--includedir=/usr/include/bind9' '--with-tuning=large' '--with-libidn2' '--with-maxminddb' '--with-gssapi=yes' '--with-lmdb=yes' '--with-json-c' '--enable-dnstap' '--with-cmocka' '--enable-fixed-rrset' '--enable-full-report' 'CPPFLAGS= -DOPENSSL_API_COMPAT=10100' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CC=gcc' 'CFLAGS=-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer' 'LDFLAGS=-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes ' 'LT_SYS_LIBRARY_PATH=/usr/lib64:' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'
compiled by GCC 13.0.1 20230401 (Red Hat 13.0.1-0)
compiled with OpenSSL version: OpenSSL 3.0.8 7 Feb 2023
linked to OpenSSL version: OpenSSL 3.0.8 7 Feb 2023
compiled with libuv version: 1.44.2
linked to libuv version: 1.44.2
compiled with libnghttp2 version: 1.52.0
linked to libnghttp2 version: 1.52.0
compiled with libxml2 version: 2.10.4
linked to libxml2 version: 21004
compiled with json-c version: 0.16
linked to json-c version: 0.16
compiled with zlib version: 1.2.13
linked to zlib version: 1.2.13
linked to maxminddb version: 1.7.1
compiled with protobuf-c version: 1.4.1
linked to protobuf-c version: 1.4.1
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): yes
default paths:
named configuration: /etc/named.conf
rndc configuration: /etc/rndc.conf
DNSSEC root key: /etc/bind.keys
nsupdate session key: /var/run/named/session.key
named PID file: /var/run/named/named.pid
named lock file: /var/run/named/named.lock
geoip-directory: /usr/share/GeoIP
```
### Steps to reproduce
- ``(sleep 6 && echo localhost.) | nslookup``
### What is the current *bug* behavior?
Starts with ``;; communications error to 127.0.0.53#53: timed out``
### What is the expected *correct* behavior?
Just like in development releases, no timeout is reported, because no timeout did occur.
### Relevant configuration files
- no configuration, just /etc/resolv.conf.
### Relevant logs and/or screenshots
- reported on https://bugzilla.redhat.com/show_bug.cgi?id=2192225
```
$ (sleep 6 && echo localhost.) | nslookup
;; communications error to 127.0.0.53#53: timed out
Server: 127.0.0.53
Address: 127.0.0.53#53
Name: localhost
Address: 127.0.0.1
Name: localhost
Address: ::1
$ (sleep 6 && echo "set d2" && echo localhost.) | nslookup 2>&1 | tee out
$ cat out
addlookup()
make_empty_lookup()
make_empty_lookup() = 0x7f7abb5b2000->references = 1
looking up localhost.
start_lookup()
setup_lookup(0x7f7abb5b2000)
resetting lookup counter.
cloning server list
clone_server_list()
make_server(127.0.0.53)
using root origin
recursive query
add_question()
starting to render the message
done rendering
create query 0x7f7abb5e3000 linked to lookup 0x7f7abb5b2000
../../../bin/dig/dighost.c:2177:lookup_attach(0x7f7abb5b2000) = 2
../../../bin/dig/dighost.c:2690:new_query(0x7f7abb5e3000) = 1
do_lookup()
start_udp(0x7f7abb5e3000)
../../../bin/dig/dighost.c:3301:query_attach(0x7f7abb5e3000) = 2
working on lookup 0x7f7abb5b2000, query 0x7f7abb5e3000
../../../bin/dig/dighost.c:3346:query_attach(0x7f7abb5e3000) = 3
udp_ready()
udp_ready(0x7f7aba5a1000, success, 0x7f7abb5e3000)
lock_lookup ../../../bin/dig/dighost.c:3188
success
../../../bin/dig/dighost.c:3189:lookup_attach(0x7f7abb5b2000) = 3
../../../bin/dig/dighost.c:3261:query_attach(0x7f7abb5e3000) = 4
recving with lookup=0x7f7abb5b2000, query=0x7f7abb5e3000, handle=0x7f7aba5a1000
recvcount=1
have local timeout of 5000
../../../bin/dig/dighost.c:3135:query_attach(0x7f7abb5e3000) = 5
sending a request
sendcount=1
../../../bin/dig/dighost.c:1761:query_detach(0x7f7abb5e3000) = 4
../../../bin/dig/dighost.c:3281:query_detach(0x7f7abb5e3000) = 3
../../../bin/dig/dighost.c:3282:lookup_detach(0x7f7abb5b2000) = 2
unlock_lookup ../../../bin/dig/dighost.c:3283
send_done(0x7f7aba5a1000, success, 0x7f7abb5e3000)
sendcount=0
lock_lookup ../../../bin/dig/dighost.c:2765
success
../../../bin/dig/dighost.c:2769:lookup_attach(0x7f7abb5b2000) = 3
../../../bin/dig/dighost.c:2787:query_detach(0x7f7abb5e3000) = 2
../../../bin/dig/dighost.c:2788:lookup_detach(0x7f7abb5b2000) = 2
check_if_done()
list empty
unlock_lookup ../../../bin/dig/dighost.c:2792
recv_done(0x7f7aba5a1000, timed out, 0x7f7abaffe0c0, 0x7f7abb5e3000)
lock_lookup ../../../bin/dig/dighost.c:3955
success
recvcount=0
../../../bin/dig/dighost.c:3960:lookup_attach(0x7f7abb5b2000) = 3
;; communications error to 127.0.0.53#53: timed out
create query 0x7f7abb5e31c0 linked to lookup 0x7f7abb5b2000
../../../bin/dig/dighost.c:2177:lookup_attach(0x7f7abb5b2000) = 4
../../../bin/dig/dighost.c:4048:new_query(0x7f7abb5e31c0) = 1
making new UDP request, 2 tries left
start_udp(0x7f7abb5e31c0)
../../../bin/dig/dighost.c:3301:query_attach(0x7f7abb5e31c0) = 2
working on lookup 0x7f7abb5b2000, query 0x7f7abb5e31c0
../../../bin/dig/dighost.c:3346:query_attach(0x7f7abb5e31c0) = 3
check_if_queries_done(0x7f7abb5b2000)
there is a pending query 0x7f7abb5e31c0
../../../bin/dig/dighost.c:4554:query_detach(0x7f7abb5e3000) = 1
../../../bin/dig/dighost.c:4562:lookup_detach(0x7f7abb5b2000) = 3
unlock_lookup ../../../bin/dig/dighost.c:4566
udp_ready()
udp_ready(0x7f7aba5a1300, success, 0x7f7abb5e31c0)
lock_lookup ../../../bin/dig/dighost.c:3188
success
../../../bin/dig/dighost.c:3189:lookup_attach(0x7f7abb5b2000) = 4
../../../bin/dig/dighost.c:3261:query_attach(0x7f7abb5e31c0) = 4
recving with lookup=0x7f7abb5b2000, query=0x7f7abb5e31c0, handle=0x7f7aba5a1300
recvcount=1
have local timeout of 5000
../../../bin/dig/dighost.c:3135:query_attach(0x7f7abb5e31c0) = 5
sending a request
sendcount=1
../../../bin/dig/dighost.c:1761:query_detach(0x7f7abb5e31c0) = 4
../../../bin/dig/dighost.c:3281:query_detach(0x7f7abb5e31c0) = 3
../../../bin/dig/dighost.c:3282:lookup_detach(0x7f7abb5b2000) = 3
unlock_lookup ../../../bin/dig/dighost.c:3283
send_done(0x7f7aba5a1300, success, 0x7f7abb5e31c0)
sendcount=0
lock_lookup ../../../bin/dig/dighost.c:2765
success
../../../bin/dig/dighost.c:2769:lookup_attach(0x7f7abb5b2000) = 4
../../../bin/dig/dighost.c:2787:query_detach(0x7f7abb5e31c0) = 2
../../../bin/dig/dighost.c:2788:lookup_detach(0x7f7abb5b2000) = 3
check_if_done()
list empty
unlock_lookup ../../../bin/dig/dighost.c:2792
recv_done(0x7f7aba5a1300, success, 0x7f7abaffad40, 0x7f7abb5e31c0)
lock_lookup ../../../bin/dig/dighost.c:3955
success
recvcount=0
../../../bin/dig/dighost.c:3960:lookup_attach(0x7f7abb5b2000) = 4
before parse starts
after parse
printmessage()
Server: 127.0.0.53
Address: 127.0.0.53#53
clone_lookup()
make_empty_lookup()
make_empty_lookup() = 0x7f7abb5b3800->references = 1
printsection()
Name: localhost
Address: 127.0.0.1
still pending.
../../../bin/dig/dighost.c:4554:query_detach(0x7f7abb5e31c0) = 1
../../../bin/dig/dighost.c:4556:_cancel_lookup()
canceling pending query 0x7f7abb5e3000, belonging to 0x7f7abb5b2000
../../../bin/dig/dighost.c:2817:query_detach(0x7f7abb5e3000) = 0
../../../bin/dig/dighost.c:2817:destroy_query(0x7f7abb5e3000) = 0
../../../bin/dig/dighost.c:1719:lookup_detach(0x7f7abb5b2000) = 3
canceling pending query 0x7f7abb5e31c0, belonging to 0x7f7abb5b2000
../../../bin/dig/dighost.c:2817:query_detach(0x7f7abb5e31c0) = 0
../../../bin/dig/dighost.c:2817:destroy_query(0x7f7abb5e31c0) = 0
../../../bin/dig/dighost.c:1719:lookup_detach(0x7f7abb5b2000) = 2
check_if_done()
list full
pending lookup 0x7f7abb5b3800
../../../bin/dig/dighost.c:4562:lookup_detach(0x7f7abb5b2000) = 1
clear_current_lookup()
lookup cleared
../../../bin/dig/dighost.c:1852:lookup_detach(0x7f7abb5b2000) = 0
destroy_lookup
freeing server 0x7f7abb595400 belonging to 0x7f7abb5b2000
start_lookup()
setup_lookup(0x7f7abb5b3800)
cloning server list
clone_server_list()
make_server(127.0.0.53)
using root origin
recursive query
add_question()
starting to render the message
done rendering
create query 0x7f7abb5e31c0 linked to lookup 0x7f7abb5b3800
../../../bin/dig/dighost.c:2177:lookup_attach(0x7f7abb5b3800) = 2
../../../bin/dig/dighost.c:2690:new_query(0x7f7abb5e31c0) = 1
do_lookup()
start_udp(0x7f7abb5e31c0)
../../../bin/dig/dighost.c:3301:query_attach(0x7f7abb5e31c0) = 2
working on lookup 0x7f7abb5b3800, query 0x7f7abb5e31c0
../../../bin/dig/dighost.c:3346:query_attach(0x7f7abb5e31c0) = 3
unlock_lookup ../../../bin/dig/dighost.c:4566
udp_ready()
udp_ready(0x7f7aba5a1180, success, 0x7f7abb5e31c0)
lock_lookup ../../../bin/dig/dighost.c:3188
success
../../../bin/dig/dighost.c:3189:lookup_attach(0x7f7abb5b3800) = 3
../../../bin/dig/dighost.c:3261:query_attach(0x7f7abb5e31c0) = 4
recving with lookup=0x7f7abb5b3800, query=0x7f7abb5e31c0, handle=0x7f7aba5a1180
recvcount=1
have local timeout of 5000
../../../bin/dig/dighost.c:3135:query_attach(0x7f7abb5e31c0) = 5
sending a request
sendcount=1
../../../bin/dig/dighost.c:1761:query_detach(0x7f7abb5e31c0) = 4
../../../bin/dig/dighost.c:3281:query_detach(0x7f7abb5e31c0) = 3
../../../bin/dig/dighost.c:3282:lookup_detach(0x7f7abb5b3800) = 2
unlock_lookup ../../../bin/dig/dighost.c:3283
send_done(0x7f7aba5a1180, success, 0x7f7abb5e31c0)
sendcount=0
lock_lookup ../../../bin/dig/dighost.c:2765
success
../../../bin/dig/dighost.c:2769:lookup_attach(0x7f7abb5b3800) = 3
../../../bin/dig/dighost.c:2787:query_detach(0x7f7abb5e31c0) = 2
../../../bin/dig/dighost.c:2788:lookup_detach(0x7f7abb5b3800) = 2
check_if_done()
list empty
unlock_lookup ../../../bin/dig/dighost.c:2792
recv_done(0x7f7aba5a1180, success, 0x7f7abaffad40, 0x7f7abb5e31c0)
lock_lookup ../../../bin/dig/dighost.c:3955
success
recvcount=0
../../../bin/dig/dighost.c:3960:lookup_attach(0x7f7abb5b3800) = 3
before parse starts
after parse
printmessage()
printsection()
Name: localhost
Address: ::1
still pending.
../../../bin/dig/dighost.c:4554:query_detach(0x7f7abb5e31c0) = 1
../../../bin/dig/dighost.c:4556:_cancel_lookup()
canceling pending query 0x7f7abb5e31c0, belonging to 0x7f7abb5b3800
../../../bin/dig/dighost.c:2817:query_detach(0x7f7abb5e31c0) = 0
../../../bin/dig/dighost.c:2817:destroy_query(0x7f7abb5e31c0) = 0
../../../bin/dig/dighost.c:1719:lookup_detach(0x7f7abb5b3800) = 2
check_if_done()
list empty
../../../bin/dig/dighost.c:4562:lookup_detach(0x7f7abb5b3800) = 1
clear_current_lookup()
lookup cleared
../../../bin/dig/dighost.c:1852:lookup_detach(0x7f7abb5b3800) = 0
destroy_lookup
freeing server 0x7f7abb595400 belonging to 0x7f7abb5b3800
start_lookup()
check_if_done()
list empty
shutting down
dighost_shutdown()
unlock_lookup ../../../bin/dig/dighost.c:4566
done, and starting to shut down
cancel_all()
lock_lookup ../../../bin/dig/dighost.c:4675
success
unlock_lookup ../../../bin/dig/dighost.c:4709
destroy_libs()
freeing task
lock_lookup ../../../bin/dig/dighost.c:4748
success
flush_server_list()
destroy DST lib
unlock_lookup ../../../bin/dig/dighost.c:4769
Removing log context
Destroy memory
```
### Possible fixes
I were unable to find exact spot where this happens. It recv_done already receives eresult==timed out, so I suspect libuv wrappers might be responsible. In master branch this seems to be fixed, happens on v9_18 only.July 2023 (9.18.17, 9.18.17-S1, 9.19.15)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4039bind 9.18.14 with error: task.c:805: INSIST((task->events).head != (event)) f...2023-05-10T11:41:24Zcarlos-paniagobind 9.18.14 with error: task.c:805: INSIST((task->events).head != (event)) failed, back trace<pre>
# named -V
BIND 9.18.14 (Extended Support Version) <id:>
running on FreeBSD amd64 12.4-STABLE FreeBSD 12.4-STABLE SOL
built by make with '--disable-linux-caps' '--localstatedir=/var' '--sysconfdir=/usr/local/etc/namedb' '--with-d...<pre>
# named -V
BIND 9.18.14 (Extended Support Version) <id:>
running on FreeBSD amd64 12.4-STABLE FreeBSD 12.4-STABLE SOL
built by make with '--disable-linux-caps' '--localstatedir=/var' '--sysconfdir=/usr/local/etc/namedb' '--with-dlopen=yes' '--with-libxml2' '--with-openssl=/usr' '--enable-dnsrps' '--with-readline=libedit' '--enable-dnstap' '--disable-fixed-rrset' '--disable-geoip' '--without-maxminddb' '--without-gssapi' '--with-libidn2=/usr/local' '--with-json-c' '--disable-largefile' '--with-lmdb=/usr/local' '--disable-querytrace' '--enable-tcp-fastopen' '--prefix=/usr/local' '--mandir=/usr/local/man' '--disable-silent-rules' '--infodir=/usr/local/share/info/' '--build=amd64-portbld-freebsd12.4' 'build_alias=amd64-portbld-freebsd12.4' 'CC=cc' 'CFLAGS=-O2 -pipe -DLIBICONV_PLUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing ' 'LDFLAGS= -L/usr/local/lib -ljson-c -fstack-protector-strong ' 'LIBS=-L/usr/local/lib' 'CPPFLAGS=-DLIBICONV_PLUG -isystem /usr/local/include' 'CPP=cpp' 'PKG_CONFIG=pkgconf' 'PKG_CONFIG_LIBDIR=/usr/ports/dns/bind918/work/.pkgconfig:/usr/local/libdata/pkgconfig:/usr/local/share/pkgconfig:/usr/libdata/pkgconfig' 'PYTHON=/usr/local/bin/python3.9' 'READLINE_CFLAGS=-L/usr/local/lib'
compiled by CLANG FreeBSD Clang 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
compiled with OpenSSL version: OpenSSL 1.1.1t-freebsd 7 Feb 2023
linked to OpenSSL version: OpenSSL 1.1.1t-freebsd 7 Feb 2023
compiled with libuv version: 1.44.2
linked to libuv version: 1.44.2
compiled with libnghttp2 version: 1.52.0
linked to libnghttp2 version: 1.52.0
compiled with libxml2 version: 2.10.3
linked to libxml2 version: 21003
compiled with json-c version: 0.16
linked to json-c version: 0.16
compiled with zlib version: 1.2.13
linked to zlib version: 1.2.13
compiled with protobuf-c version: 1.4.1
linked to protobuf-c version: 1.4.1
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): no
default paths:
named configuration: /usr/local/etc/namedb/named.conf
rndc configuration: /usr/local/etc/namedb/rndc.conf
DNSSEC root key: /usr/local/etc/namedb/bind.keys
nsupdate session key: /var/run/named/session.key
named PID file: /var/run/named/pid
named lock file: /var/run/named/named.lock
-------------------------------------------
Commands
# nslookup
> set q=any
> server ns1.rnp.br
Default server: ns1.rnp.br
Address: 200.133.59.172#53
> 0-63.254.129.200.in-addr.arpa
;; Connection to 200.133.59.172#53(200.133.59.172) for 0-63.254.129.200.in-addr.arpa failed: timed out.
;; Connection to 200.133.59.172#53(200.133.59.172) for 0-63.254.129.200.in-addr.arpa failed: connection refused.
;; Connection to 200.133.59.172#53(200.133.59.172) for 0-63.254.129.200.in-addr.arpa failed: connection refused.
task.c:805: INSIST((task->events).head != (event)) failed, back trace
0x8002be98e <isc_assertion_setcallback+0x5e> at /usr/local/lib/bind-tools/libisc-9.18.14.so
0x8002be92a <isc_assertion_failed+0xa> at /usr/local/lib/bind-tools/libisc-9.18.14.so
0x8002de2de <isc_task_run+0x43e> at /usr/local/lib/bind-tools/libisc-9.18.14.so
0x8002adc92 <isc__nmsocket_log_tls_session_reuse+0x3c2> at /usr/local/lib/bind-tools/libisc-9.18.14.so
0x8002a8223 <isc__nm_maybe_enqueue_ievent+0xa3> at /usr/local/lib/bind-tools/libisc-9.18.14.so
0x8002adb16 <isc__nmsocket_log_tls_session_reuse+0x246> at /usr/local/lib/bind-tools/libisc-9.18.14.so
0x8002a41ed <isc__netmgr_create+0x6dd> at /usr/local/lib/bind-tools/libisc-9.18.14.so
0x800b4fd1a <uv_async_send+0x3da> at /usr/local/lib/libuv.so.1
0x800b610d5 <uv_cpu_info+0xd95> at /usr/local/lib/libuv.so.1
0x800b502c8 <uv_run+0x1e8> at /usr/local/lib/libuv.so.1
0x8002a42db <isc__netmgr_create+0x7cb> at /usr/local/lib/bind-tools/libisc-9.18.14.so
0x8002e6f76 <isc__trampoline_run+0x16> at /usr/local/lib/bind-tools/libisc-9.18.14.so
Abort
</pre>June 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)Arаm SаrgsyаnArаm Sаrgsyаnhttps://gitlab.isc.org/isc-projects/bind9/-/issues/4038"tcp:checking that BIND 9 doesn't crash on long TCP messages" gets named oom-...2024-03-13T12:53:25ZMichal Nowak"tcp:checking that BIND 9 doesn't crash on long TCP messages" gets named oom-killed### Summary
TCP test consumes excessive amounts of memory, which is not accounted for in memory statistics.
### BIND version used
* ~"Affects v9.19": 1b0e7e7a50733366dfae39de93f85f2293b725ff
* ~"Affects v9.18": ff3d25a47f9f969669b2e4f...### Summary
TCP test consumes excessive amounts of memory, which is not accounted for in memory statistics.
### BIND version used
* ~"Affects v9.19": 1b0e7e7a50733366dfae39de93f85f2293b725ff
* ~"Affects v9.18": ff3d25a47f9f969669b2e4f5cde10c50f9cdd171
Versions tested but NOT affected:
* ~"v9.16": adb71afffec615f63fb79306ae429fa55a6d5fd7
### Steps to reproduce
* Run test step "tcp:checking that BIND 9 doesn't crash on long TCP messages" in a loop. Essentially:
```
for I in $(seq 1 10); do perl ../packet.pl -a "127.0.0.1" -p "53" -t tcp -r 300000 1996-alloc_dnsbuf-crash-test.pkt; sleep 5; done
```
Source:
https://gitlab.isc.org/isc-projects/bind9/-/blob/1b0e7e7a50733366dfae39de93f85f2293b725ff/bin/tests/system/tcp/tests.sh#L197
### What is the current *bug* behavior?
Kernel shows excessive memory consumption and at the same time memory statistics in BIND do not show it. Apparently it is really consumed by something because sometimes OOM killer comes to the rescue.
Job [#3348347](https://gitlab.isc.org/isc-projects/bind9/-/jobs/3348347) failed for 0f25d62a408c75820e6db585950b121fd1ffdb70.
```
I:tcp:checking that BIND 9 doesn't crash on long TCP messages (10)
I:tcp:sending 300000 time(s): 00010000000100000000000003697363036f72670000fc0001
I:tcp:failed
I:tcp:exit status: 1
I:tcp:stopping servers
I:tcp:ns1 died before a SIGTERM was sent
I:tcp:ns1 didn't die when sent a SIGTERM
I:tcp:stopping servers failed
R:tcp:FAIL
```
After a local investigation, this is a likely case of running out of memory. This is what I get locally:
```
oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-3.scope,task=lt-named,pid=68053,uid=1000
Out of memory: Killed process 68053 (lt-named) total-vm:6015604kB, anon-rss:3451292kB, file-rss:5652kB, shmem-rss:0kB, UID:1000 pgtables:10744kB oom_score_adj:0
```
~~We should bump VM's memory to more than [4G](https://gitlab.isc.org/isc-projects/gitlab-runner-scripts/-/blob/main/libvirt-qemu/domain-template.xml#L8), or rather don't run this particular test on QCOW2 images.~~September 2023 (9.16.44, 9.16.44-S1, 9.18.19, 9.18.19-S1, 9.19.17)Artem BoldarievArtem Boldarievhttps://gitlab.isc.org/isc-projects/bind9/-/issues/40379.18.13 crashed after run a few days2023-05-05T08:55:02Zshipei.xu9.18.13 crashed after run a few days
### Summary
After two days of running, my 9.18.13 version of named received a SIGSEGV signal.
### BIND version used
```
BIND 9.18.13 (Extended Support Version) <id:>
[root@ip-X-X-X-X named]# /data/named/sbin/named -V
BIND 9.18.13 (Ext...
### Summary
After two days of running, my 9.18.13 version of named received a SIGSEGV signal.
### BIND version used
```
BIND 9.18.13 (Extended Support Version) <id:>
[root@ip-X-X-X-X named]# /data/named/sbin/named -V
BIND 9.18.13 (Extended Support Version) <id:>
running on Linux x86_64 5.10.130-118.517.amzn2.x86_64 #1 SMP Wed Jul 13 16:51:52 UTC 2022
built by make with '--enable-dnstap' '--enable-epoll' '--with-json-c' '--with-libnghttp2' '--enable-doh' '--prefix=/data/named/' 'PKG_CONFIG_PATH=:/usr/local/lib/pkgconfig'
compiled by GCC 7.3.1 20180712 (Red Hat 7.3.1-15)
compiled with OpenSSL version: OpenSSL 1.1.1g FIPS 21 Apr 2020
linked to OpenSSL version: OpenSSL 1.1.1g FIPS 21 Apr 2020
compiled with libuv version: 1.39.0
linked to libuv version: 1.39.0
compiled with libnghttp2 version: 1.41.0
linked to libnghttp2 version: 1.41.0
compiled with json-c version: 0.11
linked to json-c version: 0.11
compiled with zlib version: 1.2.7
linked to zlib version: 1.2.7
compiled with protobuf-c version: 1.0.2
linked to protobuf-c version: 1.0.2
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): yes
default paths:
named configuration: /data/named/etc/named.conf
rndc configuration: /data/named/etc/rndc.conf
DNSSEC root key: /data/named/etc/bind.keys
nsupdate session key: /data/named/var/run/named/session.key
named PID file: /data/named/var/run/named/named.pid
named lock file: /data/named/var/run/named/named.lock
```
### Steps to reproduce
This is my build environment, I didn't do anything and it just crashed.
### What is the current *bug* behavior?
Named received a SIGSEGV signal and crashed.
### What is the expected *correct* behavior?
Runs normally.
### Relevant configuration files
```
tls test-tls {
key-file "/ssl_cert/star.key";
cert-file "/ssl_cert/star.pem";
dhparam-file "/ssl_cert/dhparam.pem";
ciphers "HIGH:!kRSA:!aNULL:!eNULL:!RC4:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!SHA1:!SHA256:!SHA384";
prefer-server-ciphers yes;
session-tickets no;
};
http local {
endpoints { "/dns-query"; };
};
options {
listen-on port 53 { any; };
listen-on tls test-tls { any; };
listen-on tls test-tls http local { any; };
listen-on-v6 { none; };
directory "/var/named/";
dump-file "/var/named/data/cache_dump.db";
session-keyfile "/var/named/run/session.key";
bindkeys-file "/etc/bind.keys";
key-directory "/etc";
version none;
notify no;
servfail-ttl 30;
allow-query { any; };
allow-query-cache { any; };
forward first;
hostname none;
reuseport yes;
max-cache-size 6g;
recursion yes;
querylog no;
http-streams-per-connection 100000;
http-listener-clients 100000;
recursive-clients 65535;
clients-per-query 100000;
max-clients-per-query 150000;
tcp-clients 80000;
tcp-initial-timeout 30;
tcp-idle-timeout 50;
tcp-keepalive-timeout 50;
minimal-responses no-auth;
minimal-any yes;
dnstap {
client query;
client response;
resolver query;
resolver response;
};
dnstap-output file "/var/log/dns.tap";
dnstap-identity none;
dnstap-version none;
allow-new-zones yes;
new-zones-directory "/dns-root/";
dnssec-validation no;
};
view "any" {
match-clients { any; };
allow-query-cache { any; };
max-cache-size 256m;
prefetch 10;
max-ncache-ttl 300;
forwarders { *.*.*.* port 5533; };
dlz "file system zone" {
database "dlopen /lib/dlz_filesystem_dynamic.so /dns-root/ .dns .xfr 0 ~";
};
};
```
### Relevant logs and/or screenshots
```
[root@ip-172-16-7-111 named]# gdb /data/named/sbin/named ./core.22530
GNU gdb (GDB) Red Hat Enterprise Linux 8.0.1-36.amzn2.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /data/named/sbin/named...done.
[New LWP 22532]
[New LWP 22530]
[New LWP 22531]
[New LWP 22533]
[New LWP 22534]
[New LWP 22535]
[New LWP 22541]
warning: Could not load shared library symbols for /lib/dlz_filesystem_dynamic.so.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/data/named/sbin/named -u named -c /etc/named.conf -t /data/named/chroot/'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f3c3f30f062 in SSL_SESSION_list_remove.isra.0 () from /lib64/libssl.so.1.1
[Current thread is 1 (Thread 0x7f3c3a5ff700 (LWP 22532))]
Missing separate debuginfos, use: debuginfo-install glibc-2.26-60.amzn2.x86_64 json-c-0.11-4.amzn2.0.4.x86_64 keyutils-libs-1.5.8-3.amzn2.0.2.x86_64 krb5-libs-1.15.1-37.amzn2.2.4.x86_64 libcap-2.54-1.amzn2.0.1.x86_64 libcom_err-1.42.9-19.amzn2.x86_64 libgcc-7.3.1-15.amzn2.x86_64 libnghttp2-1.41.0-1.amzn2.x86_64 libselinux-2.5-12.amzn2.0.2.x86_64 libstdc++-7.3.1-15.amzn2.x86_64 libuv-1.39.0-1.amzn2.x86_64 openssl11-libs-1.1.1g-12.amzn2.0.9.x86_64 pcre-8.32-17.amzn2.0.2.x86_64 protobuf-c-1.0.2-3.amzn2.0.1.x86_64 sssd-client-1.16.5-10.amzn2.10.x86_64 zlib-1.2.7-19.amzn2.0.1.x86_64
(gdb) bt
#0 0x00007f3c3f30f062 in SSL_SESSION_list_remove.isra.0 () from /lib64/libssl.so.1.1
#1 0x00007f3c3f30fc4b in timeout_cb () from /lib64/libssl.so.1.1
#2 0x00007f3c3ef78165 in OPENSSL_LH_doall_arg () from /lib64/libcrypto.so.1.1
#3 0x00007f3c3f310977 in SSL_CTX_flush_sessions () from /lib64/libssl.so.1.1
#4 0x00007f3c3f3297dc in tls_construct_new_session_ticket () from /lib64/libssl.so.1.1
#5 0x00007f3c3f31bbba in state_machine () from /lib64/libssl.so.1.1
#6 0x00007f3c3f309000 in SSL_do_handshake () from /lib64/libssl.so.1.1
#7 0x00007f3c410570a3 in tls_try_handshake (sock=sock@entry=0x7f3c2463fe00, presult=presult@entry=0x7f3c3a5ea070) at netmgr/tlsstream.c:340
#8 0x00007f3c41057d50 in tls_do_bio (sock=0x7f3c2463fe00, received_data=0x7f3c3a5fa0c0, send_data=0x0, finish=false) at netmgr/tlsstream.c:457
#9 0x00007f3c410174bc in isc__nm_async_readcb (worker=worker@entry=0x0, ev0=ev0@entry=0x7f3c3a5fa0f0) at netmgr/netmgr.c:2890
#10 0x00007f3c41017609 in isc__nm_readcb (sock=sock@entry=0x7f3c2478dc00, uvreq=<optimized out>, eresult=eresult@entry=ISC_R_SUCCESS) at netmgr/netmgr.c:2863
#11 0x00007f3c4101c22a in isc__nm_tcp_read_cb (stream=<optimized out>, nread=80, buf=0x7f3c3a5fa1c0) at netmgr/tcp.c:904
#12 0x00007f3c3e5c8aff in uv.read () from /lib64/libuv.so.1
#13 0x00007f3c3e5c9736 in uv.stream_io () from /lib64/libuv.so.1
#14 0x00007f3c3e5ced96 in uv.io_poll () from /lib64/libuv.so.1
#15 0x00007f3c3e5bf1a3 in uv_run () from /lib64/libuv.so.1
#16 0x00007f3c41019563 in nm_thread (worker0=0x7f3c3c0c15b8) at netmgr/netmgr.c:698
#17 0x00007f3c4104e765 in isc__trampoline_run (arg=0x7f3c3c032330) at trampoline.c:189
#18 0x00007f3c3db4c44b in start_thread () from /lib64/libpthread.so.0
#19 0x00007f3c3d88756f in clone () from /lib64/libc.so.6
(gdb)
```
### Possible fixesJune 2023 (9.16.42, 9.16.42-S1, 9.18.16, 9.18.16-S1, 9.19.14)https://gitlab.isc.org/isc-projects/bind9/-/issues/4032Cannot change max-zone-ttl for dnssec-policy insecure2023-08-01T13:02:07ZKimmo SuominenCannot change max-zone-ttl for dnssec-policy insecure<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confident...<!--
If the bug you are reporting is potentially security-related - for example,
if it involves an assertion failure or other crash in `named` that can be
triggered repeatedly - then please make sure that you make the new issue
confidential!
-->
### Summary
The `insecure` DNSSEC policy cannot be successfully applied to a zone that contains TTLs larger than 1 day (86400). The zone will not be loaded due to the `max-zone-ttl` of `P1D` that apparently is part of the `insecure` policy.
### BIND version used
```
BIND 9.16.37-Debian (Extended Support Version) <id:2b2afb2>
running on Linux x86_64 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21)
built by make with '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/x86_64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--enable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=/usr' '--with-libidn2' '--with-json-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' '--enable-dnstap' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -ffile-prefix-map=/build/bind9-t8MKLi/bind9-9.16.37=. -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
compiled by GCC 10.2.1 20210110
compiled with OpenSSL version: OpenSSL 1.1.1n 15 Mar 2022
linked to OpenSSL version: OpenSSL 1.1.1n 15 Mar 2022
compiled with libuv version: 1.40.0
linked to libuv version: 1.40.0
compiled with libxml2 version: 2.9.10
linked to libxml2 version: 20910
compiled with json-c version: 0.15
linked to json-c version: 0.15
compiled with zlib version: 1.2.11
linked to zlib version: 1.2.11
linked to maxminddb version: 1.5.2
compiled with protobuf-c version: 1.3.3
linked to protobuf-c version: 1.3.3
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): yes
default paths:
named configuration: /etc/bind/named.conf
rndc configuration: /etc/bind/rndc.conf
DNSSEC root key: /etc/bind/bind.keys
nsupdate session key: //run/named/session.key
named PID file: //run/named/named.pid
named lock file: //run/named/named.lock
geoip-directory: /usr/share/GeoIP
```
### Steps to reproduce
Change the configuration to apply `dnssec-policy insecure;` instead of a previous DNSSEC policy to a zone that has TTLs higher than 1 day.
### What is the current *bug* behavior?
Zone is not loaded due to the offending TTL.
### What is the expected *correct* behavior?
Either `max-zone-ttl unlimited;` should apply to `dnssec-policy insecure;`, or there needs to be a way to configure `max-zone-ttl` for the `insecure` policy.
### Relevant configuration files
```
zone "5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa" {
type master;
file "pri/2001.470.28.d85.rev";
dnssec-policy insecure;
inline-signing yes;
parental-agents {
"8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa";
};
notify yes;
allow-transfer {
"allow-transfer";
};
};
```
### Relevant logs and/or screenshots
KSK seen withdrawn here, `dnssec-policy insecure;` has already been applied:
```
Apr 21 10:33:46 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): reconfiguring zone keys
Apr 21 10:33:46 grendel named[2305]: keymgr: retire DNSKEY 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/RSASHA256/22324 (KSK)
Apr 21 10:33:46 grendel named[2305]: keymgr: retire DNSKEY 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/RSASHA256/45816 (ZSK)
Apr 21 10:33:46 grendel named[2305]: DNSKEY 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/RSASHA256/22324 (KSK) is now inactive
Apr 21 10:33:46 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): next key event: 21-Apr-2023 11:33:46.003
Apr 21 10:33:46 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): checkds: empty DS response from 216.218.130.2#53
Apr 21 10:33:46 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): checkds: empty DS response from 216.66.1.2#53
Apr 21 10:33:46 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): checkds: empty DS response from 216.218.131.2#53
Apr 21 10:33:46 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): checkds: empty DS response from 216.218.132.2#53
Apr 21 10:33:46 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): checkds: empty DS response from 216.66.80.18#53
Apr 21 10:33:46 grendel named[2305]: keymgr: checkds DS for key 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/RSASHA256/22324 seen withdrawn at Fri Apr 21 10:33:46 2023
Apr 21 10:33:46 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): reconfiguring zone keys
Apr 21 10:33:46 grendel named[2305]: keymgr: retire DNSKEY 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/RSASHA256/22324 (KSK)
Apr 21 10:33:46 grendel named[2305]: keymgr: retire DNSKEY 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/RSASHA256/45816 (ZSK)
Apr 21 10:33:46 grendel named[2305]: DNSKEY 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/RSASHA256/22324 (KSK) is now inactive
Apr 21 10:33:46 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): next key event: 22-Apr-2023 12:33:46.583
```
Everything is still working here:
```
Apr 21 11:53:26 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): checkds: set 5 parentals
Apr 21 11:53:26 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): reconfiguring zone keys
Apr 21 11:53:26 grendel named[2305]: keymgr: retire DNSKEY 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/RSASHA256/22324 (KSK)
Apr 21 11:53:26 grendel named[2305]: keymgr: retire DNSKEY 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/RSASHA256/45816 (ZSK)
Apr 21 11:53:26 grendel named[2305]: DNSKEY 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/RSASHA256/22324 (KSK) is now inactive
Apr 21 11:53:26 grendel named[2305]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): next key event: 22-Apr-2023 12:33:46.725
```
Restarting here:
```
Apr 21 23:37:41 grendel named[2305]: received control channel command 'stop'
Apr 21 23:37:41 grendel named[2305]: no longer listening on 127.0.0.1#53
Apr 21 23:37:41 grendel named[2305]: no longer listening on <IPv4>#53
Apr 21 23:37:41 grendel named[2305]: no longer listening on ::1#53
Apr 21 23:37:41 grendel named[2305]: no longer listening on <Global IPv6>#53
Apr 21 23:37:41 grendel named[2305]: no longer listening on <Link Local IPv6>%2#53
Apr 21 23:37:41 grendel named[2305]: shutting down: flushing changes
Apr 21 23:37:41 grendel named[2305]: stopping command channel on 127.0.0.1#953
Apr 21 23:37:41 grendel named[2305]: exiting
Apr 21 23:37:41 grendel named[137244]: starting BIND 9.16.37-Debian (Extended Support Version) <id:2b2afb2>
```
With the restart, signed versions of zones failed to load due to DNSKEY records unexpectedly now having a TTL of 7 days:
```
Apr 21 23:37:41 grendel named[137244]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): checkds: set 5 parentals
Apr 21 23:37:41 grendel named[137244]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (unsigned): loaded serial 2021091700
Apr 21 23:37:41 grendel named[137244]: dns_master_load: TTL 604800 exceeds configured max-zone-ttl 86400
Apr 21 23:37:41 grendel named[137244]: dns_master_load: out of range
Apr 21 23:37:41 grendel named[137244]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): loading from master file pri/2001.470.28.d85.rev.signed failed: out of range
Apr 21 23:37:41 grendel named[137244]: zone 5.8.d.0.8.2.0.0.0.7.4.0.1.0.0.2.ip6.arpa/IN (signed): not loaded due to errors.
```
### Possible fixesAugust 2023 (9.16.43, 9.16.43-S1, 9.18.18, 9.18.18-S1, 9.19.16)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/bind9/-/issues/4027NSEC3 of removed empty-non-terminal remains in chain, breaking validation tools2024-03-06T20:46:25ZLibor PeltanNSEC3 of removed empty-non-terminal remains in chain, breaking validation tools### Summary
A zone is being signed by an authoritative Bind9, including NSEC3 chain management. The zone contains a non-authoritative (opt-outed) NS record below an empty-non-terminal (e.g. `nonauth.ent.example.com. NS` when `ent.exampl...### Summary
A zone is being signed by an authoritative Bind9, including NSEC3 chain management. The zone contains a non-authoritative (opt-outed) NS record below an empty-non-terminal (e.g. `nonauth.ent.example.com. NS` when `ent.example.com.` has no records). I noticed that Bind9 created a NSEC3 for such empty-non-terminal (`ent.example.com.`) although it's not necessary (but not entirely wrong as well, opt-out is always optional). The problem is, that if a dynamic DDNS update to the zone removes the underlying record, implicitly removing the empty-non-terminal as well, Bind9 does not remove this NSEC3 record, which is already clearly wrong.
After such operation, indeed, both `dnssec-verify` and `ldns-verify-zone` complain about this superfluous NSEC3 record. KnotDNS's zone-checking utilities don't, but we are going to fix them to check this as well (existing NSEC3 not belonging to any node in the zone). If the user uses such verifications tool to guard his signing processes, his zone may get blocked due to this bug, until he somehow forces Bind9 to reconstruct the NSEC3-chain completely.
### BIND version used
```
BIND 9.18.12-0ubuntu0.22.04.1-Ubuntu (Extended Support Version) <id:>
running on Linux x86_64 5.15.0-70-generic #77-Ubuntu SMP Tue Mar 21 14:02:37 UTC 2023
built by make with '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' '--libdir=/usr/lib/x86_64-linux-gnu' '--sysconfdir=/etc/bind' '--with-python=python3' '--localstatedir=/' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--disable-static' '--with-gost=no' '--with-openssl=/usr' '--with-gssapi=yes' '--with-libidn2' '--with-json-c' '--with-lmdb=/usr' '--with-gnu-ld' '--with-maxminddb' '--with-atf=no' '--enable-ipv6' '--enable-rrl' '--enable-filter-aaaa' '--disable-native-pkcs11' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -ffile-prefix-map=/build/bind9-RW6AWX/bind9-9.18.12=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fno-strict-aliasing -fno-delete-null-pointer-checks -DNO_VERSION_DATE -DDIG_SIGCHASE' 'LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
compiled by GCC 11.3.0
compiled with OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
linked to OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
compiled with libuv version: 1.43.0
linked to libuv version: 1.43.0
compiled with libnghttp2 version: 1.43.0
linked to libnghttp2 version: 1.43.0
compiled with libxml2 version: 2.9.13
linked to libxml2 version: 20913
compiled with json-c version: 0.15
linked to json-c version: 0.15
compiled with zlib version: 1.2.11
linked to zlib version: 1.2.11
linked to maxminddb version: 1.5.2
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): yes
default paths:
named configuration: /etc/bind/named.conf
rndc configuration: /etc/bind/rndc.conf
DNSSEC root key: /etc/bind/bind.keys
nsupdate session key: //run/named/session.key
named PID file: //run/named/named.pid
named lock file: //run/named/named.lock
geoip-directory: /usr/share/GeoIP
```
### Steps to reproduce
* prepare a zone with a NS record below an empty-non-terminal
* configure Bind9 for automatic signing with NSEC3 opt-out
* remove this record with DDNS
* observe that the NSEC3 record belonging to the empty-non-terminal remained in the signed zone
### What is the current *bug* behavior?
Redundant NSEC3 present (orphaned) in the zone.
### What is the expected *correct* behavior?
All zone verifications tools (incl `dnssec-verify`) report no errors.
### Relevant configuration files
If you really like me to attach machine-generated configuratoin files and zones from KnotDNS testing environment, please let me know. I didn't bother to prepare a minimized reproducer, sorry.
### Relevant logs and/or screenshots
See above.
### Possible fixes
I'm not really sure if the issue happens as well if the record removed is authoritative and not opt-outed. If not, one possible fix may be based on always opt-outing the empty-non-terminals, that only contain opt-outed delegations beneath them. But a straightforward fix may be also good. I really do understand that this will be difficult :)May 2023 (9.16.41, 9.16.41-S1, 9.18.15, 9.18.15-S1, 9.19.13)Mark AndrewsMark Andrews