Without libcap, named tries to acquire new Linux capabilities (and fails)

On Linux, without libcap, named may try to expand its permitted capabilities at startup, fail, and give a misleading error message.

When named starts, it immediatly drops capabilities to a restricted set. Once a number of initialization tasks have been performed, it drops even more capabilities, before running the server. These operations happen in bin/name/unix/os.c, where the first set of permission is called initialprivs, and the second one minprivs.

Note: Since it is impossible for a process to add a capability to its own permitted set, minprivs is a subset of initialprivs.

When compiled with libcap, dropping capabilities entails:

• computing the intersection of the current permitted capability set, and the desired set
• setting the permitted and effective capability set to that intersection

In particular, when named starts, if one of the capabilities in initialprivs is missing from the permitted set, named will not try to acquire that capability. It makes sense, since it would not be allowed to (see Note above). named keeps running without the missing capability, and if that capability ends up not being used, the server functions properly.

On the other hand, when compiled without libcap, named does not lookup the permitted capability set, and attempts to set the permitted and effective capability sets to initialprivs. If any of the capabilities are missing from the permitted set, the capset system call fails, as expected, and named gives the partially misleading error message:

named: syscall(capset) failed: Operation not permitted: please ensure that the capset kernel module is loaded. see insmod(8)

This issue is encountered when trying to run named in docker. The CAP_DAC_READ_SEARCH, and CAP_SYS_RESOURCE capabilities are not present in docker by default, but are part of the initialprivs set. For example, see the bug report at https://bugs.alpinelinux.org/issues/4513.

The capabilities in initialprivs represent all the potential capabilities required by named to perform initialization. Not every run of named uses all of them. For example, CAP_DAC_READ_SEARCH, mentioned above, is only required when trying to read a configuration file owned by a non-root user and non-world-readable on startup.

The behaviour without libcap should replicate the correct behaviour implemented with libcap. This way, the capset system call would not try to expand its permitted capabilities set and fail. As a result, the error message would be displayed in fewer cases, and the information about the likely missing capset kernel module would be relevant.

It looks like the issue ended up in the code base as follows:

1. In the initial version available on gitlab (9b2267b5), without libcap, the capabilities are dropped at start up to CAP_NET_BIND_SERVICE. If that capability is not present in the permitted set, the user receives the error message:

named: syscall(capset) failed: Operation not permitted

2. A few years later, the error message on failure of the capset system call is expanded to its current form (c93003b0). It likely reflected a common issue at the time. That error message was shared by the libcap version when it was later added (0415ca35).

3. Over time, more capabilities are added to the initialprivs required set, and the problem of trying to acquire capabilities not in the permitted set gets noticed. It is fixed only in the libcap version (88674be6).