Skip to content

assertion failure: acl.c:505: INSIST(__v > 0 && __v < 0xffffffffU) failed / REQUIRE(isc_refcount_current(&dacl->references) == 0) failed

Summary

A local IP address change implicitly changes built-in localhost ACL and this can trigger crash in BIND while it is processing requests.

BIND version affected

This issue happens intermittently and it can take minutes before it crashes, so it's hard to be certain without code inspection.

  • Affects v9.20: v9.20.1
  • Other versions might be affected as well
tos-res: {3} /usr/pkg/sbin/named -V
BIND 9.20.1 (Stable Release) <id:ef7201b>
running on NetBSD amd64 10.0 NetBSD 10.0 (GENERIC) #0: Mon Jun  3 22:41:16 CEST 2024  he@tos-res.uninett.no:/usr/obj/sys/arch/amd64/compile/GENERIC
built by make with '--with-lmdb=no' '--without-gssapi' '--enable-dnstap' '--with-libxml2' '--with-json-c=no' '--with-readline' '--sysconfdir=/usr/pkg/etc' '--localstatedir=/var' '--with-openssl=/usr' '--disable-tracing' '--prefix=/usr/pkg' '--build=x86_64--netbsd' '--host=x86_64--netbsd' '--mandir=/usr/pkg/man' '--enable-option-checking=yes' 'build_alias=x86_64--netbsd' 'host_alias=x86_64--netbsd' 'CC=gcc' 'CFLAGS=-O2 -pthread -I/usr/include -I/usr/pkg/include' 'LDFLAGS=-lxml2 -Wl,-zrelro -pthread -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib' 'LIBS=' 'CPPFLAGS=-I/usr/include -I/usr/pkg/include' 'PKG_CONFIG=/usr/pkg/bin/pkg-config' 'PKG_CONFIG_PATH=' 'PKG_CONFIG_LIBDIR=/usr/pkg/lib/pkgconfig:/usr/pkg/share/pkgconfig'
compiled by GCC 10.5.0
compiled with OpenSSL version: OpenSSL 3.0.12 24 Oct 2023
linked to OpenSSL version: OpenSSL 3.0.12 24 Oct 2023
compiled with libuv version: 1.48.0
linked to libuv version: 1.48.0
compiled with liburcu version: 0.12.1
compiled with libnghttp2 version: 1.62.1
linked to libnghttp2 version: 1.62.1
compiled with libxml2 version: 2.12.8
linked to libxml2 version: 21208
compiled with zlib version: 1.2.13
linked to zlib version: 1.2.13
compiled with protobuf-c version: 1.5.0
linked to protobuf-c version: 1.5.0
threads support is enabled
DNSSEC algorithms: RSASHA1 NSEC3RSASHA1 RSASHA256 RSASHA512 ECDSAP256SHA256 ECDSAP384SHA384 ED25519 ED448
DS algorithms: SHA-1 SHA-256 SHA-384
HMAC algorithms: HMAC-MD5 HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384 HMAC-SHA512
TKEY mode 2 support (Diffie-Hellman): no
TKEY mode 3 support (GSS-API): no

default paths:
  named configuration:  /usr/pkg/etc/named.conf
  rndc configuration:   /usr/pkg/etc/rndc.conf
  nsupdate session key: /var/run/named/session.key
  named PID file:       /var/run/named/named.pid
tos-res: {4}

Steps to reproduce

It can take couple minutes to trigger, depending on how race conditions turn out. Tested on Arch Linux system:

  1. Config example: named.conf
  2. Start BIND server with multiple threads: named -n 64 -g -c named.conf
  3. Simulate legitimate clients using command yes '. A' | dnsperf -S1 -c 256
  4. Generate IP address changes which affect localhost ACL. E.g.
while true; do for OP in add del; do for I in $(seq 1 254); do sudo ip addr $OP 10.53.0.$I/32 dev lo; done; done; done

What is the current bug behavior?

BIND crashed with this assertion failure:

named[1]: acl.c:505: INSIST(__v > 0 && __v < 0xffffffffU) failed

or

named[1]: acl.c:497: REQUIRE(isc_refcount_current(&dacl->references) == 0) failed

What is the expected correct behavior?

Not crashing with an assertion failure would be preferable :)

Relevant logs

This can manifest in several backtraces.

This was the clue to reproducing it:

(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0)
    at pthread_kill.c:44
#1  0x00007e0466f76463 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x00007e0466f1d120 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007e0466f044c3 in __GI_abort () at abort.c:79
#4  0x00007e046806b53f in isc_assertion_failed (file=file@entry=0x7e0467fca021 "acl.c", line=line@entry=497, 
    type=type@entry=isc_assertiontype_require, 
    cond=cond@entry=0x7e0467fd5d98 "isc_refcount_current(&dacl->references) == 0") at assertions.c:49
#5  0x00007e0467e342fc in dns__acl_destroy (dacl=0x7e042d4f9af0) at acl.c:497
#6  dns_acl_unref (ptr=0x7e042d4f9af0) at acl.c:505
#7  0x00007e0467e347a0 in dns_aclenv_set (env=0x7e04648208c0, localhost=<optimized out>, localnets=<optimized out>)
    at acl.c:667
#8  0x00007e0467dcb143 in do_scan (mgr=mgr@entry=0x7e046494b040, verbose=verbose@entry=false, 
    config=config@entry=false) at interfacemgr.c:1357
#9  0x00007e0467dcb898 in ns_interfacemgr_scan (mgr=mgr@entry=0x7e046494b040, verbose=verbose@entry=false, 
    config=config@entry=false) at interfacemgr.c:1376
#10 0x00007e0467dcba9c in route_recv (handle=0x7e0461bff600, eresult=ISC_R_SUCCESS, region=0x7ffe51c60ce0, 
    arg=0x7e046494b040) at interfacemgr.c:247
#11 0x00007e04680574d8 in isc___nm_readcb (arg=0x7e04649b0e00) at netmgr/netmgr.c:1861
#12 isc__nm_readcb (sock=sock@entry=0x7e04611df000, uvreq=uvreq@entry=0x7e04649b0e00, 
    eresult=eresult@entry=ISC_R_SUCCESS, async=async@entry=false) at netmgr/netmgr.c:1876
#13 0x00007e0468069cb5 in isc__nm_udp_read_cb (handle=<optimized out>, nrecv=76, buf=0x7ffe51c60db0, 
    addr=<optimized out>, flags=0) at netmgr/udp.c:589
#14 0x00007e04679b1286 in uv__udp_recvmsg (handle=0x7e04611df2c8) at src/unix/udp.c:267
#15 uv__udp_io (loop=<optimized out>, w=0x7e04611df348, revents=1) at src/unix/udp.c:142
#16 0x00007e04679b230a in uv__io_poll (loop=0x7e04648ee1e0, timeout=<optimized out>) at src/unix/linux.c:1528
#17 0x00007e046799a09f in uv_run (loop=loop@entry=0x7e04648ee1e0, mode=mode@entry=UV_RUN_DEFAULT)
    at src/unix/core.c:448
#18 0x00007e046807ef1d in loop_thread (arg=0x7e04648ee1c0) at loop.c:288
#19 0x00005b57296f6977 in main (argc=<optimized out>, argv=<optimized out>) at main.c:1575
(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x000074b9db976463 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x000074b9db91d120 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x000074b9db9044c3 in __GI_abort () at abort.c:79
#4  0x00005f321b77eaca in assertion_failed (file=<optimized out>, line=505, type=isc_assertiontype_insist, cond=0x74b9dc9d5af0 "__v > 0 && __v < (4294967295U)") at main.c:234
#5  0x000074b9dca7a53a in isc_assertion_failed (file=file@entry=0x74b9dc9ca021 "acl.c", line=line@entry=505, type=type@entry=isc_assertiontype_insist, cond=cond@entry=0x74b9dc9d5af0 "__v > 0 && __v < (4294967295U)")
    at assertions.c:48
#6  0x000074b9dc8339ad in dns_acl_ref (ptr=<optimized out>) at acl.c:505
#7  dns_acl_ref (ptr=<optimized out>) at acl.c:505
#8  0x000074b9dc8339cd in dns_acl_attach (ptr=0x74b9a1f88d90, ptrp=0x74b9b03f99b0) at acl.c:505
#9  0x000074b9dc833b2f in dns_aclelement_match (reqaddr=reqaddr@entry=0x74b9b03f9c00, reqsigner=reqsigner@entry=0x0, e=e@entry=0x74b9d9284f00, env=env@entry=0x74b9d92208c0, matchelt=matchelt@entry=0x0) at acl.c:416
#10 0x000074b9dc833dc1 in dns_acl_match (reqaddr=reqaddr@entry=0x74b9b03f9c00, reqsigner=reqsigner@entry=0x0, acl=acl@entry=0x74b9d9351370, env=env@entry=0x74b9d92208c0, match=<optimized out>, matchelt=<optimized out>) at acl.c:203
#11 0x000074b9dc834015 in dns_acl_match_port_transport (reqaddr=reqaddr@entry=0x74b9b03f9c00, local_port=<optimized out>, transport=transport@entry=isc_nm_udpsocket, encrypted=encrypted@entry=false, reqsigner=reqsigner@entry=0x0, 
    acl=acl@entry=0x74b9d9351370, env=0x74b9d92208c0, match=0x74b9b03f9bcc, matchelt=0x0) at acl.c:264
#12 0x000074b9dc7c2df4 in ns_client_checkaclsilent (client=client@entry=0x74b9a6257400, netaddr=0x74b9b03f9c00, netaddr@entry=0x0, acl=0x74b9d9351370, default_allow=default_allow@entry=true) at client.c:2692
#13 0x000074b9dc7c4c05 in ns_client_request_continue (arg=arg@entry=0x74b9a6257400) at client.c:2382
#14 0x000074b9dc7c57c4 in ns_client_request (handle=<optimized out>, eresult=<optimized out>, region=<optimized out>, arg=<optimized out>) at client.c:2130
#15 0x000074b9dca664d8 in isc___nm_readcb (arg=0x74b9a63e8580) at netmgr/netmgr.c:1861
#16 isc__nm_readcb (sock=sock@entry=0x74b9d5b7d6a0, uvreq=uvreq@entry=0x74b9a63e8580, eresult=eresult@entry=ISC_R_SUCCESS, async=async@entry=false) at netmgr/netmgr.c:1876
#17 0x000074b9dca78cb5 in isc__nm_udp_read_cb (handle=<optimized out>, nrecv=17, buf=0x74b9b03fa7b0, addr=<optimized out>, flags=8) at netmgr/udp.c:589
#18 0x000074b9dc3c0fc5 in uv__udp_recvmmsg (handle=handle@entry=0x74b9d5b7d968, buf=buf@entry=0x74b9b03fb090) at src/unix/udp.c:195
#19 0x000074b9dc3c1173 in uv__udp_recvmsg (handle=0x74b9d5b7d968) at src/unix/udp.c:238
#20 uv__udp_io (loop=<optimized out>, w=0x74b9d5b7d9e8, revents=1) at src/unix/udp.c:142
#21 0x000074b9dc3c230a in uv__io_poll (loop=0x74b9d9305900, timeout=<optimized out>) at src/unix/linux.c:1528
#22 0x000074b9dc3aa09f in uv_run (loop=loop@entry=0x74b9d9305900, mode=mode@entry=UV_RUN_DEFAULT) at src/unix/core.c:448
#23 0x000074b9dca8df1d in loop_thread (arg=arg@entry=0x74b9d93058e0) at loop.c:288
#24 0x000074b9dcaa03b6 in thread_body (wrap=0x74b9d9262c80) at thread.c:85
#25 thread_run (wrap=0x74b9d9262c80) at thread.c:100
#26 0x000074b9db97439d in start_thread (arg=<optimized out>) at pthread_create.c:447
#27 0x000074b9db9f949c in __GI___clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x000074e92b34c463 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x000074e92b2f3120 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x000074e92b2da4c3 in __GI_abort () at abort.c:79
#4  0x00006416c5fa9aca in assertion_failed (file=<optimized out>, line=505, type=isc_assertiontype_insist, cond=0x74e92c1d5af0 "__v > 0 && __v < (4294967295U)") at main.c:234
#5  0x000074e92c40e53a in isc_assertion_failed (file=file@entry=0x74e92c1ca021 "acl.c", line=line@entry=505, type=type@entry=isc_assertiontype_insist, cond=cond@entry=0x74e92c1d5af0 "__v > 0 && __v < (4294967295U)")
    at assertions.c:48
#6  0x000074e92c0339ad in dns_acl_ref (ptr=<optimized out>) at acl.c:505
#7  dns_acl_ref (ptr=<optimized out>) at acl.c:505
#8  0x000074e92c0339cd in dns_acl_attach (ptr=0x74e8f17b4920, ptrp=0x74e902ff89f0) at acl.c:505
#9  0x000074e92c033b2f in dns_aclelement_match (reqaddr=reqaddr@entry=0x74e902ff8c40, reqsigner=reqsigner@entry=0x0, e=e@entry=0x74e9284fbe00, env=env@entry=0x74e928c208c0, matchelt=matchelt@entry=0x0) at acl.c:416
#10 0x000074e92c033dc1 in dns_acl_match (reqaddr=reqaddr@entry=0x74e902ff8c40, reqsigner=reqsigner@entry=0x0, acl=acl@entry=0x74e928d51370, env=env@entry=0x74e928c208c0, match=<optimized out>, matchelt=<optimized out>) at acl.c:203
#11 0x000074e92c034015 in dns_acl_match_port_transport (reqaddr=reqaddr@entry=0x74e902ff8c40, local_port=<optimized out>, transport=transport@entry=isc_nm_udpsocket, encrypted=encrypted@entry=false, reqsigner=reqsigner@entry=0x0, 
    acl=acl@entry=0x74e928d51370, env=0x74e928c208c0, match=0x74e902ff8c0c, matchelt=0x0) at acl.c:264
#12 0x000074e92c39bdf4 in ns_client_checkaclsilent (client=client@entry=0x74e8ffc65c00, netaddr=0x74e902ff8c40, netaddr@entry=0x0, acl=0x74e928d51370, default_allow=default_allow@entry=true) at client.c:2692
#13 0x000074e92c3a62a1 in query_checkcacheaccess (client=client@entry=0x74e8ffc65c00, name=name@entry=0x74e8ffd17600, qtype=qtype@entry=1, options=...) at query.c:855
#14 0x000074e92c3a6497 in query_getcachedb (client=client@entry=0x74e8ffc65c00, name=name@entry=0x74e8ffd17600, qtype=qtype@entry=1, dbp=dbp@entry=0x74e902ff9c60, options=options@entry=...) at query.c:1329
#15 0x000074e92c3ad06b in query_getdb (client=0x74e8ffc65c00, name=0x74e8ffd17600, qtype=<optimized out>, options=..., zonep=zonep@entry=0x74e902ff9cb0, dbp=dbp@entry=0x74e902ff9c60, versionp=0x74e902ff9c68, 
    is_zonep=0x74e902ff9806) at query.c:1438
#16 0x000074e92c3bb021 in ns__query_start (qctx=qctx@entry=0x74e902ff97d0) at query.c:5676
#17 0x000074e92c3bb92e in query_setup (client=client@entry=0x74e8ffc65c00, qtype=qtype@entry=29929) at query.c:5521
#18 0x000074e92c3beafc in ns_query_start (client=client@entry=0x74e8ffc65c00, handle=<optimized out>) at query.c:12154
#19 0x000074e92c39dcec in ns_client_request_continue (arg=arg@entry=0x74e8ffc65c00) at client.c:2454
#20 0x000074e92c39e7c4 in ns_client_request (handle=<optimized out>, eresult=<optimized out>, region=<optimized out>, arg=<optimized out>) at client.c:2130
#21 0x000074e92c3fa4d8 in isc___nm_readcb (arg=0x74e8ffd19e00) at netmgr/netmgr.c:1861
#22 isc__nm_readcb (sock=sock@entry=0x74e92537bc60, uvreq=uvreq@entry=0x74e8ffd19e00, eresult=eresult@entry=ISC_R_SUCCESS, async=async@entry=false) at netmgr/netmgr.c:1876
#23 0x000074e92c40ccb5 in isc__nm_udp_read_cb (handle=<optimized out>, nrecv=17, buf=0x74e902ffa7b0, addr=<optimized out>, flags=8) at netmgr/udp.c:589
#24 0x000074e92bd47fc5 in uv__udp_recvmmsg (handle=handle@entry=0x74e92537bf28, buf=buf@entry=0x74e902ffb090) at src/unix/udp.c:195
#25 0x000074e92bd48173 in uv__udp_recvmsg (handle=0x74e92537bf28) at src/unix/udp.c:238
#26 uv__udp_io (loop=<optimized out>, w=0x74e92537bfa8, revents=1) at src/unix/udp.c:142
#27 0x000074e92bd4930a in uv__io_poll (loop=0x74e928d03580, timeout=<optimized out>) at src/unix/linux.c:1528
#28 0x000074e92bd3109f in uv_run (loop=loop@entry=0x74e928d03580, mode=mode@entry=UV_RUN_DEFAULT) at src/unix/core.c:448
#29 0x000074e92c421f1d in loop_thread (arg=arg@entry=0x74e928d03560) at loop.c:288
#30 0x000074e92c4343b6 in thread_body (wrap=0x74e928c62bc0) at thread.c:85
#31 thread_run (wrap=0x74e928c62bc0) at thread.c:100
#32 0x000074e92b34a39d in start_thread (arg=<optimized out>) at pthread_create.c:447
#33 0x000074e92b3cf49c in __GI___clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
Edited by Petr Špaček
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information