GitLab

allow dns_journal_rollforward() to read old journal files

Closes #2505

See merge request !4720

tests that version 1 journal files containing version 1 transaction
headers are rolled forward correctly on server startup, then updated
into version 2 journals. also checks journal file consistency and
'max-journal-size' behavior.

'named-journalprint -x' now prints the journal's index table and
the offset of each transaction in the journal, so that index consistency
can be confirmed.

named-journalprint can now upgrade or downgrade a journal file
in place; the '-u' option upgrades and the '-d' option downgrades.

when the 'max-ixfr-ratio' option was added, journal transaction
headers were revised to include a count of RR's in each transaction.
this made it impossible to read old journal files after an upgrade.

this branch restores the ability to read version 1 transaction
headers. when rolling forward, printing journal contents, if
the wrong transaction header format is found, we can switch.

when dns_journal_rollforward() detects a version 1 transaction
header, it returns DNS_R_RECOVERABLE.  this triggers zone_postload()
to force a rewrite of the journal file in the new format, and
also to schedule a dump of the zone database with minimal delay.
journal repair is done by dns_journal_compact(), which rewrites
the entire journal, ignoring 'max-journal-size'. journal size is
corrected later.

newly created journal files now have "BIND LOG V9.2" in their headers
instead of "BIND LOG V9". files with the new version string cannot be
read using the old transaction header format. note that this means
newly created journal files will be rejected by older versions of named.

named-journalprint now takes a "-x" option, causing it to print
transaction header information before each delta, including its
format version.

Call isc__initialize()/isc__shutdown() from win32 DllMain

See merge request !4759

Call the libisc isc__initialize() constructor and isc__shutdown()
destructor from DllMain instead of having duplicate code between
those and DllMain() code.

Improve allocation-deallocation tracking with AddressSanitizer

See merge request !4748

When AddressSanitizer is in use, disable the internal mempool
implementation and redirect the isc_mempool_get to isc_mem_get
(and similarly for isc_mempool_put).  This is the method recommended
by the AddressSanitizer authors for tracking allocations and
deallocations instead of custom poison/unpoison code (see
https://github.com/google/sanitizers/wiki/AddressSanitizerManualPoisoning).

Resolve "BIND 9.16 unit tests failing reliably on x86_64 NUMA machines"

Closes #2396

See merge request !4687

The pthread_self(), thrd_current() or GetCurrentThreadId() could
actually be a pointer, so we should rather convert the value into
uintptr_t instead of unsigned long.

Convert the isc_hp API to use the globally available isc_tid_v instead
of locally defined tid_v.  This should solve most of the problems on
machines with many number of cores / CPUs.

The current isc_hp API uses internal tid_v variable that gets
incremented for each new thread using hazard pointers.  This tid_v
variable is then used as a index to global shared table with hazard
pointers state.  Since the tid_v is only incremented and never
decremented the table could overflow very quickly if we create set of
threads for short period of time, they finish the work and cease to
exist.  Then we create identical set of threads and so on and so on.
This is not a problem for a normal `named` operation as the set of
threads is stable, but the problematic place are the unit tests where we
test network manager or other APIs (task, timer) that create threads.

This commits adds a thin wrapper around any function called from
isc_thread_create() that adds unique-but-reusable small digit thread id
that can be used as index to f.e. hazard pointer tables.  The trampoline
wrapper ensures that the thread ids will be reused, so the highest
thread_id number doesn't grow indefinitely when threads are created and
destroyed and then created again.  This fixes the hazard pointer table
overflow on machines with many cores. [GL #2396]

Resolve "New stale-answer-client-timeout crashes BIND 9.16 and 9.17"

Closes #2503

See merge request !4714

When a staleonly lookup doesn't find a satisfying answer, it should
not try to respond to the client.

This is not true when the initial lookup is staleonly (that is when
'stale-answer-client-timeout' is set to 0), because no resolver fetch
has been created at this point. In this case continue with the lookup
normally.

Fix a crash that can happen in the following scenario:

A client request is received. There is no data for it in the cache,
(not even stale data). A resolver fetch is created as part of
recursion.

Some time later, the fetch still hasn't completed, and
stale-answer-client-timeout is triggered. A staleonly lookup is
started. It will also find no data in the cache.

So 'query_lookup()' will call 'query_gotanswer()' with ISC_R_NOTFOUND,
so this will call 'query_notfound()' and this will start recursion.

We will eventually end up in 'ns_query_recurse()' and that requires
the client query fetch to be NULL:

    REQUIRE(client->query.fetch == NULL);

If the previously started fetch is still running this assertion
fails.

The crash is easily prevented by not requiring recursion for
staleonly lookups.

Also remove a redundant setting of the staleonly flag at the end of
'query_lookup_staleonly()' before destroying the query context.

Add a system test to catch this case.

Resolve "Regression in BIND 9.16.10, DNSSEC fails due to improper NSEC3 creation witihin named"

Closes #2498

See merge request !4739

When applying dnssec-policy on a dynamic zone (e.g. that allows Dynamic
Updates), the NSEC3 parameters were put on the queue, but they were
not being processed (until a reload of the zone or reconfiguration).

Process the NSEC3PARAM queue on zone postload when handling a
dynamic zone.

GitLab issue #2498 is a bug report on NSEC3 with dynamic zones. Tests
for it in the nsec3 system test directory were missing.

Resolve "CID 320483:  API usage errors  (LOCK)"

Closes #2507

See merge request !4721

Also address race between reading and testing mpctx->allocated
and incrementing mpctx->allocated.

Disable safe-guard assertion in DLL_THREAD_ATTACH/DLL_THREAD_DETACH

Closes #2519

See merge request !4738

The BIND 9 libraries on Windows define DllMain() optional entry point
into a dynamic-link library (DLL).  When the system starts or terminates
a process or thread, it calls the entry-point function for each loaded
DLL using the first thread of the process.

When the DLL is being loaded into the virtual address space of the
current process as a result of the process starting up, we make a call
to DisableThreadLibraryCalls() which should disable the
DLL_THREAD_ATTACH and DLL_THREAD_DETACH notifications for the specified
dynamic-link library (DLL).

This seems not be the case because we never check the return value of
the DisableThreadLibraryCalls() call, and it could in fact fail.  The
DisableThreadLibraryCalls() function fails if the DLL specified by
hModule has active static thread local storage, or if hModule is an
invalid module handle.

In this commit, we remove the safe-guard assertion put in place for the
DLL_THREAD_ATTACH and DLL_THREAD_DETACH events and we just ignore them.
BIND 9 doesn't create/destroy enough threads for it actually to make any
difference, and in fact we do use static thread local storage in the
code.

add placeholder

See merge request !4737

Initialize checknames field in dns_view_create()

Closes #2446

See merge request !4706

The 'checknames' field wasn't initialized in dns_view_create(), but it
should otherwise AddressSanitizer identifies the following runtime error
in query_test.c.

    runtime error: load of value 190, which is not a valid value for type '_Bool'

This reverts commit c75484c4.

'checknames' field of struct dns_view is not initialized by
dns_view_create(). ASAN identified this as runtime error:

    runtime error: load of value 190, which is not a valid value for type '_Bool'

Add Alpine Linux 3.13

See merge request !4724

Do not remove stderr from pict output

See merge request !4727

Removing stderr from the pict tool serves no purpose and drops valuable
information, we might use when debugging failed pairwise CI job, such
as:

    Input Error: A parameter names must be unique

Resolve "CID 320481:  Null pointer dereferences  (REVERSE_INULL)"

Closes #2508

See merge request !4722

    *** CID 320481:  Null pointer dereferences  (REVERSE_INULL)
    /bin/tests/wire_test.c: 261 in main()
    255     			process_message(input);
    256     		}
    257     	} else {
    258     		process_message(input);
    259     	}
    260
       CID 320481:  Null pointer dereferences  (REVERSE_INULL)
       Null-checking "input" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
    261     	if (input != NULL) {
    262     		isc_buffer_free(&input);
    263     	}
    264
    265     	if (printmemstats) {
    266     		isc_mem_stats(mctx, stdout);

Resolve "CID 281450: Dereference before null check (REVERSE_INULL)"

Closes #2493

See merge request !4684