Be more patient when stopping servers in the system tests

See merge request !6847

When the TCP test is run on the busy server, the server might take a
while to wind the server down because it might still be processing all
that 300k invalid XFR requests.

Increate the rncd wait time to 120 seconds, the SIGTERM time to 300
seconds, and reduce the time to wait for ans servers from 1200 second
to just 120 seconds.

Don't stop timer from dns__nta_shutdown()

Closes #3571

See merge request !6844

The dns__nta_shutdown() could be run from different threads and it was
accessing nta->timer unlocked.  Don't check and stop the timer from
dns__nta_shutdown() directly, but leave it for the async callback.

Refactor isc_ratelimiter API

Closes #3555

See merge request !6842

Because the dns_zonemgr_create() was run before the loopmgr was started,
the isc_ratelimiter API was more complicated that it had to be.  Move
the dns_zonemgr_create() to run_server() task which is run on the main
loop, and simplify the isc_ratelimiter API implementation.

The isc_timer is now created in the isc_ratelimiter_create() and
starting the timer is now separate async task as is destroying the timer
in case it's not launched from the loop it was created on.  The
ratelimiter tick now doesn't have to create and destroy timer logic and
just stops the timer when there's no more work to do.

This should also solve all the races that were causing the
isc_ratelimiter to be left dangling because the timer was stopped before
the last reference would be detached.

The isc_ratelimiter API was missing unit tests.

Add a new set of unit tests for the isc_ratelimiter API.

The detach function in the ISC_REFCOUNT_IMPL macro was missing DbC
checks, add them.

Resolve "dns_message_checksig under oss_fuzz is not seeing the data files"

Closes #3569

See merge request !6840

Access to the source tree is not available with oss_fuzz.  Have
fuzz/dns_message_checksig build and populate a key directory for
the fuzzer to use.  This contains a key pair and a zone file which
has the public key from the key pair.  Clean it up on shutdown.

Remove debugging fprintf from run_server()

See merge request !6843

Ondřej Surý authored Sep 29, 2022 and Ondřej Surý committed Sep 29, 2022

In the loopmgr branch, we forgot the scissors^Hdebugging output in the
patient^Hnamed, remove it.

Improve DBC in isc_mem_free

See merge request !6839

Unlike standard free(), isc_mem_free() is not a no-op when passed a
NULL pointer. For size accounting purposes it calls sallocx(), which
crashes when passed a NULL pointer. To get more helpful diagnostics,
REQUIRE() that the pointer is not NULL so that when the programmer
makes a mistake they get a backtrace that shows what went wrong.

Call the isc__nm_udp_send() callbacks asynchronously on shutdown

Closes #3534

See merge request !6828

Extra care must be taken when executing the callbacks to prevent the
deadlocks on the caller's side.  Add a paragraph that addresses when we
can and when we cannot call the callbacks directly.

The isc__nm_udp_send() callback would be called synchronously when
shutting down or when the socket has been closed.  This could lead to
double locking in the calling code and thus those callbacks needs to be
called asynchronously.

Resolve "XoT breaks DDNS update forwarding"

Closes #3512

See merge request !6710

Add several test cases in the 'upforwd' system test to make sure
that different scenarios of Dynamic DNS update forwarding are
tested, in particular when both the original and forwarded requests
are over Do53, or DoT, or they use different transports.

Now that the 'dns_request' supports using TLS transport, implement
dynamic update forwarding using DoT when the primary server is
configured to use a TLS transport.

Previously, when using such configuration, the dynamic update forwarding
feature was broken.

Resolve "suffix may be used before being assigned in qmin/ans3/ans.py"

Closes #3562

See merge request !6811

    CID 350722 (#5 of 7): Bad use of null-like value (FORWARD_NULL)
        12. invalid_operation: Invalid operation on null-like value suffix.
    145        r.authority.append(
    146            dns.rrset.from_text(
    147                "icky.ptang.zoop.boing." + suffix,
    148                1,
    149                IN,
    150                NS,
    151                "a.bit.longer.ns.name." + suffix,
    152            )
    153        )

Resolve "Missing RSA_free call in opensslrsa_verify2"

Closes #3551

See merge request !6810

Use custom isc_mem based allocator for libuv, OpenSSL, and libxml2

Closes #3559

See merge request !6807

There's a known memory leak in the engine_pkcs11 at the time of writing
this and it interferes with the named ability to check for memory leaks
in the OpenSSL memory context by default.

Add an autoconf option to explicitly enable the memory leak detection,
and use it in the CI except for pkcs11 enabled builds.  When this gets
fixed in the engine_pkc11, the option can be enabled by default.

As we can't check the deallocations done in the library memory contexts
by default because it would always fail on non-clean exit (that happens
on error or by calling exit() early), we just want to enable the checks
to be done on normal exit.

The libxml2 library provides a way to replace the default allocator with
user supplied allocator (malloc, realloc, strdup and free).

Create a memory context specifically for libxml2 to allow tracking the
memory usage that has originated from within libxml2.  This will provide
a separate memory context for libxml2 to track the allocations and when
shutting down the application it will check that all libxml2 allocations
were returned to the allocator.

Additionally, move the xmlInitParser() and xmlCleanupParser() calls from
bin/named/main.c to library constructor/destructor in libisc library.

The OpenSSL library provides a way to replace the default allocator with
user supplied allocator (malloc, realloc, and free).

Create a memory context specifically for OpenSSL to allow tracking the
memory usage that has originated from within OpenSSL.  This will provide
a separate memory context for OpenSSL to track the allocations and when
shutting down the application it will check that all OpenSSL allocations
were returned to the allocator.

The libuv library provides a way to replace the default allocator with
user supplied allocator (malloc, realloc, calloc and free).

Create a memory context specifically for libuv to allow tracking the
memory usage that has originated from within libuv.  This requires
libuv >= 1.38.0 which provides uv_library_shutdown() function that
assures no more allocations will be made.