Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
BIND
BIND
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 637
    • Issues 637
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 105
    • Merge Requests 105
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • ISC Open Source Projects
  • BINDBIND
  • Issues
  • #2210

Closed
Open
Opened Oct 09, 2020 by nan@wang🚴🏽

BIND 9.11.19 dead lock

Summary

Seems that BIND has a deadlock. and I don't find any related race condition issues, maybe it is a new issue?

BIND version used

BIND 9.11.19-RedHat-9.11.10-20200601113814.alios7 (Extended Support Version) <id:905ec64>
running on Linux x86_64 3.10.0-327.ali2012.alios7.x86_64 #1 SMP Mon Oct 9 14:09:14 CST 2017
built by make with '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--with-libtool' '--localstatedir=/var' '--enable-threads' '--enable-epoll' '--with-tuning=large' '--enable-ipv6' '--with-pic' '--disable-static' '--disable-openssl-version-check' '--with-python=/home/tops/bin/python2.7' '--with-python-install-dir=/home/tops' '--with-docbook-xsl=/usr/share/sgml/docbook/xsl-stylesheets' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CFLAGS= -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' 'LDFLAGS=-Wl,-z,relro ' 'CPPFLAGS= -DDIG_SIGCHASE' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'
compiled by GCC 4.8.5 20150623 (Red Hat 4.8.5-4)
compiled with OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
linked to OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
compiled with libxml2 version: 2.9.1
linked to libxml2 version: 20901
compiled with zlib version: 1.2.7
linked to zlib version: 1.2.7
threads support is enabled

default paths:
  named configuration:  /etc/named.conf
  rndc configuration:   /etc/rndc.conf
  DNSSEC root key:      /etc/bind.keys
  nsupdate session key: /var/run/named/session.key
  named PID file:       /var/run/named/named.pid
  named lock file:      /var/run/named/named.lock

Steps to reproduce

Not clear.

What is the current bug behavior?

BIND is hung.

What is the expected correct behavior?

(What you should see instead.)

Relevant configuration files

logging {
	channel "default_debug" {
		file "data/named.run";
		severity dynamic;
	};
};
options {
	bindkeys-file "/etc/named.iscdlv.key";
	directory "/var/named";
	dump-file "/var/named/data/cache_dump.db";
	listen-on port 53 {
		127.0.0.1/32;
	};
	listen-on-v6 port 53 {
		::1/128;
	};
	memstatistics-file "/var/named/data/named_mem_stats.txt";
	statistics-file "/var/named/data/named_stats.txt";
	dnssec-enable yes;
	dnssec-lookaside auto;
	dnssec-validation yes;
	recursion yes;
	allow-query {
		"localhost";
	};
};
zone "." IN {
	type hint;
	file "named.ca";
};
zone "localhost.localdomain" IN {
	type master;
	file "named.localhost";
	allow-update {
		"none";
	};
};
zone "localhost" IN {
	type master;
	file "named.localhost";
	allow-update {
		"none";
	};
};
zone "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" IN {
	type master;
	file "named.loopback";
	allow-update {
		"none";
	};
};
zone "1.0.0.127.in-addr.arpa" IN {
	type master;
	file "named.loopback";
	allow-update {
		"none";
	};
};
zone "0.in-addr.arpa" IN {
	type master;
	file "named.empty";
	allow-update {
		"none";
	};
};

Relevant logs and/or screenshots

Pstack result as followed:

Thread 19 (Thread 0x7f8a3d2fa700 (LWP 20008)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 18 (Thread 0x7f8a3caf9700 (LWP 20009)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 17 (Thread 0x7f8a3c2f8700 (LWP 20010)):
#0  0x00007f8a3ee3251d in __lll_lock_wait () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3ee2de51 in _L_lock_1022 () from /usr/lib64/libpthread.so.0
#2  0x00007f8a3ee2ddf2 in pthread_mutex_lock () from /usr/lib64/libpthread.so.0
#3  0x00007f8a40e20cd5 in empty_bucket (res=0x7f87d17a14b8) at resolver.c:8910
#4  0x00007f8a40e26c51 in fctx_doshutdown (task=<optimized out>, event=<optimized out>) at resolver.c:4144
#5  0x00007f8a3fafd6bb in dispatch (manager=0x7f8a41514eb0) at task.c:1157
#6  run (uap=0x7f8a41514eb0) at task.c:1331
#7  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#8  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 16 (Thread 0x7f8a3baf7700 (LWP 20011)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 15 (Thread 0x7f8a3b2f6700 (LWP 20012)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 14 (Thread 0x7f8a3aaf5700 (LWP 20013)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 13 (Thread 0x7f8a3a2f4700 (LWP 20014)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 12 (Thread 0x7f8a39af3700 (LWP 20015)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 11 (Thread 0x7f8a392f2700 (LWP 20016)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafcc33 in isc__task_beginexclusive (task0=<optimized out>) at task.c:1757
#2  0x000000000046850b in load_configuration ()
#3  0x000000000046c1d5 in loadconfig ()
#4  0x000000000046c42e in ns_server_reconfigcommand ()
#5  0x00000000004373bd in ns_control_docommand ()
#6  0x000000000043a483 in control_recvmessage ()
#7  0x00007f8a3fafd6bb in dispatch (manager=0x7f8a41514eb0) at task.c:1157
#8  run (uap=0x7f8a41514eb0) at task.c:1331
#9  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#10 0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 10 (Thread 0x7f8a38af1700 (LWP 20017)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 9 (Thread 0x7f8a382f0700 (LWP 20018)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 8 (Thread 0x7f8a37aef700 (LWP 20019)):
#0  0x00007f8a3ee3251d in __lll_lock_wait () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3ee2de51 in _L_lock_1022 () from /usr/lib64/libpthread.so.0
#2  0x00007f8a3ee2ddf2 in pthread_mutex_lock () from /usr/lib64/libpthread.so.0
#3  0x00007f8a40e253ee in dns_resolver_shutdown (res=0x7f87d17a14b8) at resolver.c:9399
#4  0x00007f8a40e650c9 in view_flushanddetach (viewp=<optimized out>, flush=<optimized out>) at view.c:601
#5  0x000000000042e387 in exit_check ()
#6  0x0000000000443310 in prefetch_done ()
#7  0x00007f8a3fafd6bb in dispatch (manager=0x7f8a41514eb0) at task.c:1157
#8  run (uap=0x7f8a41514eb0) at task.c:1331
#9  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#10 0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 7 (Thread 0x7f8a372ee700 (LWP 20020)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 6 (Thread 0x7f8a36aed700 (LWP 20021)):
#0  0x00007f8a3ee3251d in __lll_lock_wait () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3ee2de51 in _L_lock_1022 () from /usr/lib64/libpthread.so.0
#2  0x00007f8a3ee2ddf2 in pthread_mutex_lock () from /usr/lib64/libpthread.so.0
#3  0x00007f8a40e20cd5 in empty_bucket (res=0x7f87d17a14b8) at resolver.c:8910
#4  0x00007f8a40d35d9a in fetch_callback (task=<optimized out>, ev=0x7f85a1732568) at adb.c:3868
#5  0x00007f8a3fafd6bb in dispatch (manager=0x7f8a41514eb0) at task.c:1157
#6  run (uap=0x7f8a41514eb0) at task.c:1331
#7  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#8  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 5 (Thread 0x7f8a362ec700 (LWP 20022)):
#0  0x00007f8a3ee3251d in __lll_lock_wait () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3ee2de51 in _L_lock_1022 () from /usr/lib64/libpthread.so.0
#2  0x00007f8a3ee2ddf2 in pthread_mutex_lock () from /usr/lib64/libpthread.so.0
#3  0x00007f8a40e62106 in dns_view_findzonecut2 (view=0x7f88f1f49240, name=name@entry=0x7f851fd3cd08, fname=fname@entry=0x7f8a362e9070, now=now@entry=0, options=0, use_hints=use_hints@entry=true, use_cache=use_cache@entry=true, rdataset=rdataset@entry=0x7f89103158e8, sigrdataset=sigrdataset@entry=0x0) at view.c:1295
#4  0x00007f8a40e625e8 in dns_view_findzonecut (view=<optimized out>, name=name@entry=0x7f851fd3cd08, fname=fname@entry=0x7f8a362e9070, now=now@entry=0, options=<optimized out>, use_hints=use_hints@entry=true, rdataset=rdataset@entry=0x7f89103158e8, sigrdataset=sigrdataset@entry=0x0) at view.c:1256
#5  0x00007f8a40e23b62 in fctx_create (res=res@entry=0x7f87d17a14b8, name=name@entry=0x7f851fd3cd08, type=type@entry=1, domain=0x7f8a362e9070, domain@entry=0x0, nameservers=nameservers@entry=0x0, client=client@entry=0x0, options=options@entry=32, bucketnum=bucketnum@entry=400, depth=depth@entry=2, qc=qc@entry=0xf7d4918, fctxp=fctxp@entry=0x7f8a362e9f98) at resolver.c:4454
#6  0x00007f8a40e25dd8 in dns_resolver_createfetch3 (res=<optimized out>, name=name@entry=0x7f851fd3cd08, type=type@entry=1, domain=domain@entry=0x0, nameservers=nameservers@entry=0x0, forwarders=forwarders@entry=0x0, client=client@entry=0x0, id=id@entry=0, options=options@entry=32, depth=depth@entry=2, qc=qc@entry=0xf7d4918, task=0x7f8930a85ae8, action=action@entry=0x7f8a40d35c90 <fetch_callback>, arg=arg@entry=0x7f851fd3cd00, rdataset=rdataset@entry=0x7f851fd39088, sigrdataset=sigrdataset@entry=0x0, fetchp=fetchp@entry=0x7f851fd39080) at resolver.c:9630
#7  0x00007f8a40d30269 in fetch_name (adbname=adbname@entry=0x7f851fd3cd00, start_at_zone=start_at_zone@entry=false, depth=depth@entry=2, qc=qc@entry=0xf7d4918, type=type@entry=1) at adb.c:4056
#8  0x00007f8a40d39dde in dns_adb_createfind2 (adb=0x7f851fd08aa0, task=0x7f89251db3f0, action=action@entry=0x7f8a40e2b4b0 <fctx_finddone>, arg=arg@entry=0x7f88eadec120, name=name@entry=0x7f8a362eadd0, qname=qname@entry=0x7f88eadec130, qtype=28, options=223, now=now@entry=1601867731, target=target@entry=0x0, port=53, depth=2, qc=0xf7d4918, findp=findp@entry=0x7f8a362ea8b8) at adb.c:3192
#9  0x00007f8a40e1f92d in findname (fctx=fctx@entry=0x7f88eadec120, name=name@entry=0x7f8a362eadd0, port=port@entry=0, options=<optimized out>, options@entry=31, flags=flags@entry=0, now=1601867731, overquota=overquota@entry=0x7f8a362ead70, need_alternate=need_alternate@entry=0x7f8a362ead57, no_addresses=no_addresses@entry=0x7f8a362ead5c) at resolver.c:3166
#10 0x00007f8a40e27ca2 in fctx_getaddresses (fctx=fctx@entry=0x7f88eadec120, badcache=badcache@entry=false) at resolver.c:3462
#11 0x00007f8a40e2a36a in fctx_try (fctx=0x7f88eadec120, retrying=<optimized out>, badcache=<optimized out>) at resolver.c:3819
#12 0x00007f8a40e2d994 in resquery_response (task=<optimized out>, event=<optimized out>) at resolver.c:8747
#13 0x00007f8a3fafd6bb in dispatch (manager=0x7f8a41514eb0) at task.c:1157
#14 run (uap=0x7f8a41514eb0) at task.c:1331
#15 0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#16 0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 4 (Thread 0x7f8a35aeb700 (LWP 20023)):
#0  0x00007f8a3ee2f995 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fafd4af in dispatch (manager=0x7f8a41514eb0) at task.c:1103
#2  run (uap=0x7f8a41514eb0) at task.c:1331
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 3 (Thread 0x7f8a352ea700 (LWP 20024)):
#0  0x00007f8a3ee2fd42 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
#1  0x00007f8a3fb1a3c8 in isc_condition_waituntil (c=c@entry=0x7f8a4151c078, m=m@entry=0x7f8a4151c028, t=t@entry=0x7f8a4151c06c) at condition.c:59
#2  0x00007f8a3fb036b3 in run (uap=0x7f8a4151c010) at timer.c:811
#3  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#4  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 2 (Thread 0x7f8a34ae9700 (LWP 20025)):
#0  0x00007f8a3e7ec183 in epoll_wait () from /usr/lib64/libc.so.6
#1  0x00007f8a3fb112f6 in watcher (uap=0x7f8a4151e010) at socket.c:4302
#2  0x00007f8a3ee2be25 in start_thread () from /usr/lib64/libpthread.so.0
#3  0x00007f8a3e7ebbad in clone () from /usr/lib64/libc.so.6
Thread 1 (Thread 0x7f8a41555840 (LWP 20007)):
#0  0x00007f8a3e7235f2 in sigsuspend () from /usr/lib64/libc.so.6
#1  0x00007f8a3fb04f00 in isc__app_ctxrun (ctx0=ctx0@entry=0x7f8a3fd38dc0 <isc_g_appctx>) at app.c:723
#2  0x00007f8a3fb0515c in isc__app_run () at app.c:756
#3  0x00007f8a3fb05a30 in isc_app_run () at ../app_api.c:207
#4  0x000000000042a905 in main ()

Possible fixes

(If you can, link to the line of code that might be responsible for the problem.)

Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
Reference: isc-projects/bind9#2210