Implement kTLS support in BIND
Recent versions of Linux and FreeBSD support TLS encryption by kernel (kTLS). One of the benefits of it is that when TLS encryption is performed by kernel, it might use additional hardware features otherwise not available in the user space, including offloading TLS encryption to the NICs that support that (e.g. NVIDIA Mellanox ConnectX-6 Dx), almost completely freeing the CPU from this task, because even in the case of hardware acceleration of encryption within the CPU, it still requires some cycles from it. Also, using it might reduce memory copying in some cases.
Of course, kernel space encryption is more limited compared to the one provided by OpenSSL and its derivatives in the user space: these limitations are imposed by hardware - e.g. NICs might not support anything but AES 128 (aka TLS_AES_128_GCM_SHA256
), as it is the only cipher mandatory for TLS v1.3). If it is good enough for WEB servers, it should be good enough for DNS, too.
Even when kTLS is used, the handshake itself happens in the user space (e.g. using OpenSSL) with negotiated parameters passed to the kernel using setsockopt()
calls on a TCP socket descriptor.
OpenSSL provides support for kTLS encryption natively since version 3.X (see SSL_OP_ENABLE_KTLS
option) but, as far as I understand it, it does so only when OpenSSL manages the underlying TCP socket file descriptor natively: not our case, as we are using LibUV for that. However, considering that the idea of kTLS is that with it enabled, we are supposed to pass unencrypted data to send()
and recv()
, that is kTLS-enabled socket from the higher level perspective works (mostly) as a TCP socket, we might try the following approach to implement kTLS, that might work:
- We use our existing code (
tlsstream.c
) to handle handshake, just like we do now; - After completing the handshake, we pass the negotiated information to the kernel. OpenSSL might have some interfaces for that. In the worst case, we might need to do that by hand using.
setsockopt()
; - Then, we add new code paths to
tlsstream.c
to bypass TLS connection objects (isc_tls_t
) and use the underlying TCP connection directly, which, by now, works in "kTLS-mode", providing transparent TLS encryption; - Control messages, like TLS shutdown, will require additional care.
That is how I see the initial plan that might or might not work. There can (and, likely, will) be unforeseen obstacles that might turn out to overcomplicate the code base so much that it might make it unfeasible to implement, like adding a kTLS-only transport. Furthermore, that might require some assistance from LibUV. That will require some trial and error.
That is mostly written with Linux in mind. If the kTLS interface in FreeBSD is similar enough (it seems so at the first glance), we should support both platforms.
The issue is created mostly to dump the information from my mind and keep kTLS under our radar: we might want to do that, as at least dnsdist
has experimental support for it. It will be even more important in the future, as it seems now that encrypted DNS transport will be even more important to the point of replacing the good ol' Do53 at some point.
For sure, it is not a 9.20 material - rather 9.21-9.22 if we are lucky, as it is a big feature. Also, I foresee a similar concept eventually appearing for QUIC, too (kQUIC?). Also, I am aquiet certain that we will need #3504 (closed) for this (implemented here: !8576 (merged)).
See also:
- https://docs.kernel.org/networking/tls.html
- https://man.freebsd.org/cgi/man.cgi?query=ktls&apropos=0&sektion=0&manpath=FreeBSD+13.0-RELEASE+and+Ports&arch=default&format=html
-
https://delthas.fr/blog/2023/kernel-tls/ - mostly discusses it in the context of HTTP and
sendfile()
acceleration, but contains many references on the topic. - https://docs.nvidia.com/networking/display/ofedv512580/kernel+transport+layer+security+(ktls)+offloads