Resolve "RUNTIME_CHECK(res == 0) in SO_REUSE and SO_INCOMING_CPU, commit 09ba47b0"

See merge request !3635

(cherry picked from commit 80a2ef89)

The SO_INCOMING_CPU is available since Linux 3.19 for getting the value,
but only since Linux 4.4 for setting the value (see below for a full
description).  BIND 9 should not fail when setting the option on the
socket fails, as this is only an optimization and not hard requirement
to run BIND 9.

    SO_INCOMING_CPU (gettable since Linux 3.19, settable since Linux 4.4)
        Sets or gets the CPU affinity of a socket.  Expects an integer flag.

            int cpu = 1;
            setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, sizeof(cpu));

        Because all of the packets for a single stream (i.e., all
	packets for the same 4-tuple) arrive on the single RX queue that
	is associated with a particular CPU, the typical use case is to
	employ one listening process per RX queue, with the incoming
	flow being handled by a listener on the same CPU that is
	handling the RX queue.  This provides optimal NUMA behavior and
	keeps CPU caches hot.

(cherry picked from commit 4ec357da)

Merge branch '1877-reduce-default-for-max-stale-ttl-to-something-more-reasonable-12-hours-or-similar-v9_16' into 'v9_16'

Resolve "Reduce default for max-stale-ttl to something more reasonable - 12 hours or similar" (v9.16)

See merge request !3628

(cherry picked from commit e8172828)

(cherry picked from commit fc4f3b92)

Originally, the default value for max-stale-ttl was 1 week, which could
and in some scenarios lead to cache exhaustion on a busy resolvers.
Picking the default value will always be juggling between value that's
useful (e.g. keeping the already cached records after they have already
expired and the upstream name servers are down) and not bloating the
cache too much (e.g. keeping everything for a very long time).  The new
default reflects what we think is a reasonable to time to react on both
sides (upstream authoritative and downstream recursive).

(cherry picked from commit 13fd3ecf)

Content, clarity, and grammar edits to history.rst

See merge request !3633

(cherry picked from commit f6164e08)

Content, clarity, and grammar edits to troubleshooting.rst

See merge request !3632

(cherry picked from commit c6a064a6)

Content, grammar, and clarity updates to security.rst

See merge request !3631

(cherry picked from commit 7c213676)

Resolve "kasp: bug in keymgr_key_has_successor()"

See merge request !3622

When creating the successor, the current active key (predecessor)
should change its goal state to HIDDEN.

Also add two useful debug logs in the keymgr_key_rollover function.

(cherry picked from commit e71d6029)

Catch a case where if the prepublication time of the successor key
is later than the retire time of the predecessor. If that is the
case we should prepublish as soon as possible, a.k.a. now.

(cherry picked from commit c08d0f7d)

The `dns_keymgr_run()` function became quite long, put the logic
that looks if a new key needs to be created (start a key rollover)
in a separate function.

(cherry picked from commit bcf81924)

The logic in `keymgr_key_has_successor(key, keyring)` is flawed, it
returns true if there is any key in the keyring that has a successor,
while what we really want here is to make sure that the given key
has a successor in the given keyring.

Rather than relying on `keymgr_key_exists_with_state`, walk the
list of keys in the keyring and check if the key is a successor of
the given predecessor key.

(cherry picked from commit 0d578097)

Resolve "kasp: Set correct keytimings"

See merge request !3620

The usage of 'date -d' in the kasp system test is not portable,
replace with a python script.  Also remove some leftover
"set_keytime 'yes'" calls.

(cherry picked from commit 5b3decaf)

(cherry picked from commit bcf3c9fe)

This improves keytime testing on algorithm rollover.  It now
tests for specific times, and also tests for SyncPublish and
Removed keytimes.

(cherry picked from commit 61c1040a)

This improves keytime testing on reconfiguration of the
dnssec-policy.

(cherry picked from commit da5e1e3a)

This improves keytime testing on CSK rollover.  It now
tests for specific times, and also tests for SyncPublish and
Removed keytimes.

Since an "active key" for ZSK and KSK means something
different, this makes it tricky to decide when a CSK is
active. An "active key" intuitively means the key is signing
so we say a CSK is active when it is creating zone signatures.

This change means a lot of timings for the CSK rollover tests
need to be adjusted.

The keymgr code needs a slight change on calculating the
prepublication time: For a KSK we need to include the parent
registration delay, but for CSK we look at the zone signing
property and stick with the ZSK prepublication calculation.

(cherry picked from commit e2334337)

This improves keytime testing on KSK rollover.  It now
tests for specific times, and also tests for SyncPublish and
Removed keytimes.

(cherry picked from commit 649d0833)

Registration delay is not part of the Iret retire interval, thus
removed from the calculation when setting the Delete time metadata.

Include the registration delay in prepublication time, because
we need to prepublish the key sooner than just the Ipub
publication interval.

(cherry picked from commit 50bbbb76)

This improves keytime testing on ZSK rollover.  It now
tests for specific times, and also tests for SyncPublish and
Removed keytimes.

(cherry picked from commit e01fcbba)

This improves keytime testing for enabling DNSSEC.  It now
tests for specific times, and also tests for SyncPublish.

(cherry picked from commit cf51c87f)

Set the SyncPublish metadata on keys that don't have them yet.

(cherry picked from commit 30cb5c97)

This commit adds testing keytiming metadata.  In order to facilitate
this, the kasp system test undergoes a few changes:

1. When finding a key file, rather than only saving the key ID,
   also save the base filename and creation date with `key_save`.
   These can be used later to set expected key times.
2. Add a test function `set_addkeytime` that takes a key, which
   keytiming to update, a datetime in keytiming format, and a number
   (seconds) to add, and sets the new time in the given keytime
   parameter of the given key.  This is used to set the expected key
   times.
3. Split `check_keys` in `check_keys` and `check_keytimes`.  First we
   need to find the keyfile before we can check the keytimes.
   We need to retrieve the creation date (and sometimes other
   keytimes) to determine the other expected key times.
4. Add helper functions to set the expected key times per policy.
   This avoids lots of duplication.

Check for keytimes for the first test cases (all that do not cover
rollovers).

(cherry picked from commit f8e34b57)

Stop tracking in the comments the number of key parameters in the
kasp system test, it adds nothing beneficial.

(cherry picked from commit 8483f712)

After removing dnssec-settime calls that set key rollover
relationship, we can adjust the counts in test output filenames.

Also fix a couple of more wrong counts in output filenames.

(cherry picked from commit 8204e31f)

Using dnssec-setttime after dnssec-keygen in the kasp system test
can lead to off by one second failures, so reduce the usage of
dnssec-settime in the setup scripts.  This commit deals with
setting the key rollover relationship (predecessor/successor).

(cherry picked from commit 5a590c47)

In the kasp system test, we are going to set the keytimes on
dnssec-keygen so we can test them against the key creation time.
This prevents off by one second in the test, something that can
happen if you set those times with dnssec-settime after
dnssec-keygen.

Also fix some test output filenames.

(cherry picked from commit 637d5f9a)

While kasp relies on key states to determine when a key needs to
be published or be used for signing, the keytimes are used by
operators to get some expectation of key publication and usage.

Update the code such that these keytimes are set appropriately.
That means:
- Print "PublishCDS" and "DeleteCDS" times in the state files.
- The keymgr sets the "Removed" and "PublishCDS" times and derives
  those from the dnssec-policy.
- Tweak setting of the "Retired" time, when retiring keys, only
  update the time to now when the retire time is not yet set, or is
  in the future.

This also fixes a bug in "keymgr_transition_time" where we may wait
too long before zone signatrues become omnipresent or hidden. Not
only can we skip waiting the sign delay Dsgn if there is no
predecessor, we can also skip it if there is no successor.

Finally, this commit moves setting the lifetime, reducing two calls
to one.

(cherry picked from commit 18dc27af)

For testing purposes mainly, we want to allow set keytimings on
generated keys, such that we don't have to "keygen/settime" which
can result in one second off times.

(cherry picked from commit 1c216317)

Add Danger Python to GitLab CI (v9_16)

See merge request !3611

Certain rules of the BIND development process are not codified anywhere
and/or are used inconsistently.  In an attempt to improve this
situation, add a GitLab CI job which uses Danger Python to add comments
to merge requests when certain expectations are not met.  Two categories
of feedback are used, only one of which - fail() - causes the GitLab CI
job to fail.  Exclude dangerfile.py from Python QA checks as the way the
contents of that file are evaluated triggers a lot of Flake8 and PyLint
warnings.

(cherry picked from commit 36bb45a8)

backport sphinx documentation to v9_16

See merge request !3536

Also converted logging-categories.rst from a table to text and adjusted
the util/check-categories.sh script.

(cherry picked from commit 1e067c4d)