9.18.13 crashed after run a few days
Summary
After two days of running, my 9.18.13 version of named received a SIGSEGV signal.
BIND version used
BIND 9.18.13 (Extended Support Version) <id:>
[root@ip-X-X-X-X named]# /data/named/sbin/named -V
BIND 9.18.13 (Extended Support Version) <id:>
running on Linux x86_64 5.10.130-118.517.amzn2.x86_64 #1 SMP Wed Jul 13 16:51:52 UTC 2022
built by make with '--enable-dnstap' '--enable-epoll' '--with-json-c' '--with-libnghttp2' '--enable-doh' '--prefix=/data/named/' 'PKG_CONFIG_PATH=:/usr/local/lib/pkgconfig'
compiled by GCC 7.3.1 20180712 (Red Hat 7.3.1-15)
compiled with OpenSSL version: OpenSSL 1.1.1g FIPS 21 Apr 2020
linked to OpenSSL version: OpenSSL 1.1.1g FIPS 21 Apr 2020
compiled with libuv version: 1.39.0
linked to libuv version: 1.39.0
compiled with libnghttp2 version: 1.41.0
linked to libnghttp2 version: 1.41.0
compiled with json-c version: 0.11
linked to json-c version: 0.11
compiled with zlib version: 1.2.7
linked to zlib version: 1.2.7
compiled with protobuf-c version: 1.0.2
linked to protobuf-c version: 1.0.2
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): yes
TKEY mode 3 support (GSS-API): yes
default paths:
named configuration: /data/named/etc/named.conf
rndc configuration: /data/named/etc/rndc.conf
DNSSEC root key: /data/named/etc/bind.keys
nsupdate session key: /data/named/var/run/named/session.key
named PID file: /data/named/var/run/named/named.pid
named lock file: /data/named/var/run/named/named.lock
Steps to reproduce
This is my build environment, I didn't do anything and it just crashed.
What is the current bug behavior?
Named received a SIGSEGV signal and crashed.
What is the expected correct behavior?
Runs normally.
Relevant configuration files
tls test-tls {
key-file "/ssl_cert/star.key";
cert-file "/ssl_cert/star.pem";
dhparam-file "/ssl_cert/dhparam.pem";
ciphers "HIGH:!kRSA:!aNULL:!eNULL:!RC4:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!SHA1:!SHA256:!SHA384";
prefer-server-ciphers yes;
session-tickets no;
};
http local {
endpoints { "/dns-query"; };
};
options {
listen-on port 53 { any; };
listen-on tls test-tls { any; };
listen-on tls test-tls http local { any; };
listen-on-v6 { none; };
directory "/var/named/";
dump-file "/var/named/data/cache_dump.db";
session-keyfile "/var/named/run/session.key";
bindkeys-file "/etc/bind.keys";
key-directory "/etc";
version none;
notify no;
servfail-ttl 30;
allow-query { any; };
allow-query-cache { any; };
forward first;
hostname none;
reuseport yes;
max-cache-size 6g;
recursion yes;
querylog no;
http-streams-per-connection 100000;
http-listener-clients 100000;
recursive-clients 65535;
clients-per-query 100000;
max-clients-per-query 150000;
tcp-clients 80000;
tcp-initial-timeout 30;
tcp-idle-timeout 50;
tcp-keepalive-timeout 50;
minimal-responses no-auth;
minimal-any yes;
dnstap {
client query;
client response;
resolver query;
resolver response;
};
dnstap-output file "/var/log/dns.tap";
dnstap-identity none;
dnstap-version none;
allow-new-zones yes;
new-zones-directory "/dns-root/";
dnssec-validation no;
};
view "any" {
match-clients { any; };
allow-query-cache { any; };
max-cache-size 256m;
prefetch 10;
max-ncache-ttl 300;
forwarders { *.*.*.* port 5533; };
dlz "file system zone" {
database "dlopen /lib/dlz_filesystem_dynamic.so /dns-root/ .dns .xfr 0 ~";
};
};
Relevant logs and/or screenshots
[root@ip-172-16-7-111 named]# gdb /data/named/sbin/named ./core.22530
GNU gdb (GDB) Red Hat Enterprise Linux 8.0.1-36.amzn2.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /data/named/sbin/named...done.
[New LWP 22532]
[New LWP 22530]
[New LWP 22531]
[New LWP 22533]
[New LWP 22534]
[New LWP 22535]
[New LWP 22541]
warning: Could not load shared library symbols for /lib/dlz_filesystem_dynamic.so.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/data/named/sbin/named -u named -c /etc/named.conf -t /data/named/chroot/'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f3c3f30f062 in SSL_SESSION_list_remove.isra.0 () from /lib64/libssl.so.1.1
[Current thread is 1 (Thread 0x7f3c3a5ff700 (LWP 22532))]
Missing separate debuginfos, use: debuginfo-install glibc-2.26-60.amzn2.x86_64 json-c-0.11-4.amzn2.0.4.x86_64 keyutils-libs-1.5.8-3.amzn2.0.2.x86_64 krb5-libs-1.15.1-37.amzn2.2.4.x86_64 libcap-2.54-1.amzn2.0.1.x86_64 libcom_err-1.42.9-19.amzn2.x86_64 libgcc-7.3.1-15.amzn2.x86_64 libnghttp2-1.41.0-1.amzn2.x86_64 libselinux-2.5-12.amzn2.0.2.x86_64 libstdc++-7.3.1-15.amzn2.x86_64 libuv-1.39.0-1.amzn2.x86_64 openssl11-libs-1.1.1g-12.amzn2.0.9.x86_64 pcre-8.32-17.amzn2.0.2.x86_64 protobuf-c-1.0.2-3.amzn2.0.1.x86_64 sssd-client-1.16.5-10.amzn2.10.x86_64 zlib-1.2.7-19.amzn2.0.1.x86_64
(gdb) bt
#0 0x00007f3c3f30f062 in SSL_SESSION_list_remove.isra.0 () from /lib64/libssl.so.1.1
#1 0x00007f3c3f30fc4b in timeout_cb () from /lib64/libssl.so.1.1
#2 0x00007f3c3ef78165 in OPENSSL_LH_doall_arg () from /lib64/libcrypto.so.1.1
#3 0x00007f3c3f310977 in SSL_CTX_flush_sessions () from /lib64/libssl.so.1.1
#4 0x00007f3c3f3297dc in tls_construct_new_session_ticket () from /lib64/libssl.so.1.1
#5 0x00007f3c3f31bbba in state_machine () from /lib64/libssl.so.1.1
#6 0x00007f3c3f309000 in SSL_do_handshake () from /lib64/libssl.so.1.1
#7 0x00007f3c410570a3 in tls_try_handshake (sock=sock@entry=0x7f3c2463fe00, presult=presult@entry=0x7f3c3a5ea070) at netmgr/tlsstream.c:340
#8 0x00007f3c41057d50 in tls_do_bio (sock=0x7f3c2463fe00, received_data=0x7f3c3a5fa0c0, send_data=0x0, finish=false) at netmgr/tlsstream.c:457
#9 0x00007f3c410174bc in isc__nm_async_readcb (worker=worker@entry=0x0, ev0=ev0@entry=0x7f3c3a5fa0f0) at netmgr/netmgr.c:2890
#10 0x00007f3c41017609 in isc__nm_readcb (sock=sock@entry=0x7f3c2478dc00, uvreq=<optimized out>, eresult=eresult@entry=ISC_R_SUCCESS) at netmgr/netmgr.c:2863
#11 0x00007f3c4101c22a in isc__nm_tcp_read_cb (stream=<optimized out>, nread=80, buf=0x7f3c3a5fa1c0) at netmgr/tcp.c:904
#12 0x00007f3c3e5c8aff in uv.read () from /lib64/libuv.so.1
#13 0x00007f3c3e5c9736 in uv.stream_io () from /lib64/libuv.so.1
#14 0x00007f3c3e5ced96 in uv.io_poll () from /lib64/libuv.so.1
#15 0x00007f3c3e5bf1a3 in uv_run () from /lib64/libuv.so.1
#16 0x00007f3c41019563 in nm_thread (worker0=0x7f3c3c0c15b8) at netmgr/netmgr.c:698
#17 0x00007f3c4104e765 in isc__trampoline_run (arg=0x7f3c3c032330) at trampoline.c:189
#18 0x00007f3c3db4c44b in start_thread () from /lib64/libpthread.so.0
#19 0x00007f3c3d88756f in clone () from /lib64/libc.so.6
(gdb)