* digdelv test was just expecting default server EDNS buffer size to be
  4096, the test needed only slight adjustment

(cherry picked from commit f1556f8c)

The DNS Flag Day 2020 aims to remove the IP fragmentation problem from
the UDP DNS communication.  In this commit, we implement the minimal
required changes by changing the defaults for `edns-udp-size`,
`max-udp-size` and `nocookie-udp-size` to `1232` (the value picked by
DNS Flag Day 2020).

(cherry picked from commit bb990030)

Refactor the setsockopt() code in network manager into helper functions (v9.16)

See merge request !4228

The SO_REUSEADDR, SO_REUSEPORT and SO_REUSEPORT_LB has different meaning
on different platform. In this commit, we split the function to set the
reuse of address/port and setting the load-balancing into separate
functions.

The libuv library already have multiplatform support for setting
SO_REUSEADDR and SO_REUSEPORT that allows binding to the same address
and port, but unfortunately, when used after the load-balancing socket
options have been already set, it overrides the previous setting, so we
need our own helper function to enable the SO_REUSEADDR/SO_REUSEPORT
first and then enable the load-balancing socket option.

(cherry picked from commit fd975a55)

Ondřej Surý authored Oct 05, 2020 and Ondřej Surý committed Oct 05, 2020

On POSIX based systems both uv_os_sock_t and uv_os_fd_t are both typedef
to int.  That's not true on Windows, where uv_os_sock_t is SOCKET and
uv_os_fd_t is HANDLE and they differ in level of indirection.

(cherry picked from commit acb6ad9e)

Ondřej Surý authored Oct 05, 2020 and Ondřej Surý committed Oct 05, 2020

The isc__nm_socket_freebind() has been refactored to match other
isc__nm_socket_...() helper functions and take uv_os_fd_t and
sa_family_t as function arguments.

(cherry picked from commit 9dc01a63)

Ondřej Surý authored Oct 05, 2020 and Ondřej Surý committed Oct 05, 2020

This commits add isc__nm_socket_dontfrag() helper functions.

(cherry picked from commit d685bbc8)

Ondřej Surý authored Oct 05, 2020 and Ondřej Surý committed Oct 05, 2020

The setting of SO_REUSE**** and SO_INCOMING_CPU have been moved into a
separate helper functions.

(cherry picked from commit 5daaca71)

Resolve "rndc trigger rollover"

See merge request !4226

These tests don't require a specific algorithm so they should use
the DEFAULT_ALGORITHM from 'conf.sh.common'.

(cherry picked from commit 78c09f56)

It is better to add new result codes than to overload existing codes.

(cherry picked from commit 70d1ec43)

While working on 'rndc dnssec -rollover' I noticed the following
(small) issues:

- The key files where updated with hints set to "-when" and that
  should always be "now.
- The kasp system test did not properly update the test number when
  calling 'rndc dnssec -checkds' (and ensuring that works).
- There was a missing ']' in the rndc.c help output.

(cherry picked from commit edc53fc4)

When users (accidentally) try to roll an inactive key, throw an error.

(cherry picked from commit fcd34abb)

This command is similar in arguments as -checkds so refactor the
'named_server_dnssec' function accordingly.  The only difference
are that:

- It does not take a "publish" or "withdrawn" argument.
- It requires the key id to be set (add a check to make sure).

Add tests that will trigger rollover immediately and one that
schedules a test in the future.

(cherry picked from commit e826faca)

Add to the keymgr a function that will schedule a rollover. This
basically means setting the time when the key needs to retire,
and updating the key lifetime, then update the state file. The next
time that named runs the keymgr the new lifetime will be taken into
account.

(cherry picked from commit df8276ae)

Change condition for rndc dumpdb -expired

See merge request !4225

After backporting #1870 to 9.11-S I saw that the condition check there
is different than in the main branch. In 9.11-S "stale" can mean
stale and serve-stale, or not active (awaiting cleanup). In 9.16 and
later versions, "stale" is stale and serve-stale, and "ancient" means
not active (awaiting cleanup). An "ancient" RRset is one that is not
active (TTL expired) and is not eligble for serve-stale.

Update the condition for rndc dumpdb -expired to closer match what is
in 9.11-S.

(cherry picked from commit 5614454c)

Resolve "dnssec-policy behaviour for algorithm 15 keys different to algorithm 8 keys"

See merge request !4222

Sometimes, not all keys have been created in time before 'check_keys'
is called. Run a 'retry_quiet' on checking the number of keys before
continuing checking the key data.

(cherry picked from commit af3b0149)

The minimal size for RSASHA1, RSASHA256 is 512, but due to bad
assignment it was set to 1024.

(cherry picked from commit 7c555254)

This test case is unrelated to the fix for #2171 but was added to
reproduce the problem.

(cherry picked from commit 621093fe)

The 'wait_for_nsec' does not need to add TSIG because it calls
'dig_with_opts' and that already checks for TSIG.

(cherry picked from commit 43c68067)

This is a bug that needs to be noted.

(cherry picked from commit d6d41e96)

The kasp code had bad implicit size values for the cryptographic
algorithms Ed25519 and Ed448. When creating keys they would never
match the dnssec-policy, leading to new attempts to create keys.

These algorithms were previously not yet added to the system tests,
due to lack of availability on some systems.

(cherry picked from commit 0e207392)

Use the testcrypto script to see if these algorithms are supported by
openssl. If so, add the specific configuration to the named.conf file
and touch a file to indicate support. If the file exists, the
corresponding setup and tests are performed.

(cherry picked from commit 7be18357)

[v9_16] "rrset-order" fixes and cleanups

See merge request !4218

(cherry picked from commit 27c815a2)

Certain parts of the existing documentation for the "rrset-order"
statement are incorrect, others are ambiguous.  Rework the relevant
section of the ARM to make it clear and up-to-date with the source code.

(cherry picked from commit 2ac04dc9)

Make sure "order none" RRset ordering rules are tested in the
"rrsetorder" system test just like all other rule types are.  As the
check for the case of no "rrset-order" rule matching a given RRset also
tests "order none" (rather than "order random", as the test code may
suggest at first glance), replace the test code for that case so that it
matches other "order none" tests.

(cherry picked from commit abdd4c89)

named-checkconf treats the following configuration as valid:

    options {
        rrset-order {
            order none;
        };
    };

Yet, the above configuration causes named to crash on startup with:

    order.c:74: REQUIRE(mode == 0x00000800 || mode == 0x00000400 || mode == 0x00800000) failed, back trace

Add DNS_RDATASETATTR_NONE to the list of RRset ordering modes accepted
by dns_order_add() to allow "order none" to be used in "rrset-order"
rules.  This both prevents the aforementioned crashes and addresses the
discrepancy between named-checkconf and named.

(cherry picked from commit dbcf683c)

Fix the clang 12 warnings with multi-line strings in string arrays

See merge request !4212

Ondřej Surý authored Sep 16, 2020 and Ondřej Surý committed Oct 01, 2020

The clang 12 has a new warning that warns when using multi-line strings
in the string arrays, f.e.:

    { "aa",
      "b"
      "b",
      "cc" }

would generate warning like this:

    private_test.c:162:7: error: suspicious concatenation of string literals in an array initialization; did you mean to separate the elements with a comma? [-Werror,-Wstring-concatenation]
                                      "33333/RSASHA1" };
                                      ^
    private_test.c:161:7: note: place parentheses around the string literal to silence warning
                                      "Done removing signatures for key "
                                      ^
    private_test.c:197:7: error: suspicious concatenation of string literals in an array initialization; did you mean to separate the elements with a comma? [-Werror,-Wstring-concatenation]
                                      "NSEC chain",
                                      ^
    private_test.c:196:7: note: place parentheses around the string literal to silence warning
                                      "Removing NSEC3 chain 1 0 30 DEAF / creating "
                                      ^
    2 errors generated.

(cherry picked from commit 7b07f229)

Synchronize the netmgr changes with main branch

See merge request !4211

As the query_prefetch() or query_rpzfetch() could be called during
"regular" fetch, we need to introduce separate storage for attaching
the nmhandle during prefetching the records.  The query_prefetch()
and query_rpzfetch() are guarded for re-entrance by .query.prefetch
member of ns_client_t, so we can reuse the same .prefetchhandle for
both.

(cherry picked from commit d4976e0e)

Evan Hunt authored Sep 03, 2020 and Ondřej Surý committed Oct 01, 2020

(cherry picked from commit e460e321)

Evan Hunt authored Sep 05, 2020 and Ondřej Surý committed Oct 01, 2020

The LD_WRAP test in configure was broken, and failed to
indicate LD_WRAP support correctly, resulting in some unit
tests failing to run.

(cherry picked from commit cfa4ea64)

The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from
nm_destroy() has been refactored, so all use the netievents instead of
directly touching the worker structure members.  This allows us to
remove most of the locking as the .paused and .finished members are
always accessed from the matching nm_thread.

When shutting down the nm_thread(), instead of issuing uv_stop(), we
just shutdown the .async handler, so all uv_loop_t events are properly
finished first and uv_run() ends gracefully with no outstanding active
handles in the loop.

(cherry picked from commit e5ab137b)

Witold Krecicki authored Sep 02, 2020 and Ondřej Surý committed Oct 01, 2020

If NETMGR_TRACE is defined, we now maintain a list of active sockets
in the netmgr object and a list of active handles in each socket
object; by walking the list and printing `backtrace` in a debugger
we can see where they were created, to assist in in debugging of
reference counting errors.

On shutdown, if netmgr finds there are still active sockets after
waiting, isc__nm_dump_active() will be called to log the list of
active sockets and their underlying handles, along with some details
about them.

(cherry picked from commit 00e04a86)

Evan Hunt authored Jul 15, 2020 and Ondřej Surý committed Oct 01, 2020

if more than 10 seconds pass while we wait for netmgr events to
finish running on shutdown, something is almost certainly wrong
and we should assert and crash.

(cherry picked from commit 2f2d60a9)

Evan Hunt authored Jul 16, 2020 and Ondřej Surý committed Oct 01, 2020

the test server running in shutdown/resolver was not logging
any debug info, which made it difficult to diagnose test failures.

(cherry picked from commit cc7ceace)