GitLab

The custom builds (oot, asan, tsan) were mostly built using Debian sid
amd64 image.  The problem was that this image broke too easily, because
it's Debian "unstable" after all.

This commit introduces "base_image" that should be most stable with
extra bits on top (clang, coccinelle, cppcheck, ...).  Currently, that
would be Debian buster amd64.

Other changes introduced by this commit:

* Change the default clang version to 10
* Run both ASAN and TSAN with both gcc and clang compilers
* Remove Clang Debian stretch i386 job

There are several reason why remove Debian 8 from the CI:

* Debian 8 ("jessie") has been superseded by Debian 9 ("stretch").
* Regular security support updates have been discontinued as of
  June 17th, 2018.
* Jessie LTS is supported from 17th June 2018 to June 30, 2020.

In other words, it's no longer officially supported by Debian security
team, but by the volunteer/paid contributor composed LTS team.  And the
release will be discontinued in three months from now.  We can use the
freed CI resources to bring new platforms or just to make the jobs run a
bit faster.

When unit test fails, core file is created. Kyua's 'debug' command can
run GDB on it and provide backtrace. Unfortunately Kyua is picky about
location of these core files we opt to use custom Kyua fork and copy
core files from Kyua working directory to source tree and make it
available in GitLab.

The environment variable MAKE has been replaced with MAKE_COMMAND,
because overriding MAKE variable also changed the definition of the MAKE
inside the Makefiles, and we want only a single wrapper around the whole
build process.

Previously, setting `MAKE` to `bear make` meant that `bear make` would
be run at every nested make invocation, which messed up the upcoming
automake transition as compile_commands.json would be generated in every
subdirectory instead of just having one central file at the top of the
build tree.

All *:sid:amd64 jobs were errorneously copied to *:sid:arm64 including
the extra cppcheck run.  Remove the extra definitions from arm64 jobs.

All jobs now use solely the newer needs configuration to declare
dependencies between jobs:

    needs:
      - job: <foo>
        artifacts: true

instead of combination of dependencies and needs which is deprecated.
This change completely unbundles the stages (alas the stages still needs
to stay because the job graph has to stay acyclic between the stages).

To get rid of the currently used FreeBSD-specific executor, move FreeBSD
CI jobs to libvirt-based executors.  Make the necessary tag and variable
adjustments.

Since FreeBSD 12.1 is the current FreeBSD 12.x release, replace FreeBSD
12.0 GitLab CI jobs with their up-to-date counterparts.

libtool is a requirement to use automake (see GL #4), so this commit
removes the ability to compile BIND 9 without libtool.

ABI checker tools generate HTML and TXT API compatibility reports of
BIND libraries. Comparison is being done between two bind source trees
which hold built BIND.

In the CI one version is the reference version defined by
BIND_BASELINE_VERSION variable, the latter one is the HEAD of branch
under test.

OpenBSD virtual machines seem to affected particularly badly by other
activity happening on the host.  This causes trouble around release
time: when multiple tags are pushed to the repository, a large number of
jobs is started concurrently on all CI runners.  In extreme cases, this
causes the system test suite to run for about an hour (!) on OpenBSD
VMs, with multiple tests failing.  We investigated the test artifacts
for all such cases in the past and the outcome was always the same: test
failures were caused by extremely slow I/O on the guest.  We tried
various tricks to work around this problem, but nothing helped.

Given the above, stop running OpenBSD system test jobs for pending BIND
releases to prevent the results of these jobs from affecting the
assessment of a given release's readiness for publication.  This change
does not affect OpenBSD build jobs.  OpenBSD system test jobs will still
be run for scheduled and web-requested pipelines, to make sure we catch
any severe issues with test code on that platform sooner or later.

This job leverages respdiff test from the private bind-qa repo.

  - Define the SLOT environment variable before starting the test.  This
    variable defaults to 0 and that does not work with SoftHSM 2.

  - The system test expects the PIN environment variable to be set to
    "1234" while bin/tests/prepare-softhsm2.sh sets it to "0000".
    Update bin/tests/prepare-softhsm2.sh so that it sets the PIN to
    "1234".

  - Move contents of bin/tests/system/pkcs11/prereq.sh to
    bin/tests/system/pkcs11/setup.sh as the former was creating a file
    called "supported" that was getting removed by the latter before
    bin/tests/system/pkcs11/tests.sh could access it.

  - Fix typo in "have_ecx".

Artifacts generated by the docs:sid:amd64 job need to be retained longer
than for other jobs as they are used for building bind.isc.org contents.
If these artifacts are removed too quickly, pipelines in the pages/bind
GitLab project start failing, preventing content updates from being
published.  Increase lifetime of the relevant job artifacts to prevent
this from happening.

Submissions to Coverity Scan should be limited to those originated from
release branches and only from a specific schedule which holds
COVERITY_SCAN_PROJECT_NAME and COVERITY_SCAN_TOKEN variables.

This job requires two CI variables to be set:

  - COVERITY_SCAN_PROJECT_NAME: project name, which is associated with
    the BIND branch for which this job is executed, e.g. "bind-master",

  - COVERITY_SCAN_TOKEN: project token.

Since OpenBSD 6.6 is the current OpenBSD release, replace OpenBSD 6.5
GitLab CI jobs with their up-to-date counterparts.

As CI jobs for OpenBSD 6.6 will be run by a generalized libvirt executor
rather than an OpenBSD-specific one, make the necessary tag and variable
adjustments as well.

Ensure BIND is continuously tested on Tumbleweed, a pure rolling release
version of openSUSE.  This will allow BIND incompatibilities with latest
upstream versions of its dependencies to be caught more quickly.

Since Alpine Linux 3.11 is the current Alpine Linux release, replace
Alpine Linux 3.10 GitLab CI jobs with their up-to-date counterparts.

Fixes #1546.

Since Fedora 31 is the current Fedora release, replace Fedora 30 GitLab
CI jobs with their up-to-date counterparts.

Add a GitLab CI job (which is run only if all other jobs in a pipeline
succeed) that builds a BIND release tarball, i.e. fetches the source
tarball from the tarball building job, creates Windows zips, puts
certain parts of BIND documentation into the appropriate places, and
packs it all up into a single tarball whose contents can be subsequently
signed and published.

Add a system test job for binaries created by Visual Studio in the
"Debug" build configuration to GitLab CI so that they can be tested
along their "Release" counterparts when necessary.

Add a Visual Studio build job using the "Debug" build configuration to
GitLab CI without enabling it for every pipeline as it takes about twice
as long to complete as its "Release" counterpart.

Add a set of jobs to GitLab CI that create a BIND source tarball and
then build and test its contents.  Run those extra jobs only when a tag
is pushed to the Git repository as they are only meant to be sanity
checks of BIND source tarball contents.

The util/prepare-softhsm2.sh script is useful for initializing a working
SoftHSM environment which can be used by unit tests and system tests.
However, since it is a test-specific script, it does not really belong
in the util/ subdirectory which is mostly pruned during the BIND source
tarball creation process.  Move the prepare-softhsm2.sh script to
bin/tests/ so that its location is more appropriate for its purpose and
also so that it does not get removed during the BIND source tarball
creation process, allowing it to be used for setting up test
environments for tarball-based builds.