Make It Possible for BIND to Accept PROXYv2
There is a need to implement PROXYv2 protocol support in BIND
and dig
in order to make it possible for frontend proxies to carry client source and destination addresses (as well as other information) to BIND instances. Many of our friendly competitors have already done that.
The popular proxy software packages are dnsdist
and HAProxy
, but there are more of them. There are also cloud infrastructure providers that have support for PROXYv2 protocol (see Cloudflare Spectrum, Amazon Elastic Load Balancer, Azure Private Link).
How Does It Work
The general idea is that at the start of a connection, the client (which, in this case, usually means a proxying frontend like dnsdist
or HAProxy
) sends a small header with the client source and destination addresses ahead of data. As one could expect, in the case of UDP, the header is located right before the data in a datagram. The addresses in the header are meant to be used by the server instead of the connected peer and interface addresses. The headers are extensible and can contain additional information.
Additionally, the headers can contain arbitrarily defined TLV data records, with some of the record types defined in the specification. Also, specifically for the cases when the connection is not originated by a proxying frontend, there is a message format that does not contain addresses. In that case, the headers contain LOCAL
command as opposed to PROXY
(for the cases when there is information about endpoints in the headers).
Another point regarding the extensibility of the format is that it is valid to send headers with the so-called "unspecified" address format. This is meant to be used for specific cases when address types covered by the spec are not applicable. In that case, most of the PROXYv2 header can be considered an opaque blob.
The flexibility of the PROXYv2 protocol should be taken into account from the very beginning, as it would be hard to change the implementation as an afterthought. That is especially true if we plan to eventually support forwarding PROXY
headers. In that case, we should forward them as-is without attempting to reconstruct them, as in some cases, that might be impossible.
Security Considerations
PROXYv2 has only one purpose - to allow easily spoofing the source and destination addresses even for cases when otherwise that would have been complicated to do so. It has some security implications.
That means the number of clients (=proxying frontends) from which a server can accept queries over PROXYv2 must be precisely controlled and contain only authorised parties.
The Case of PROXYv2 and Encrypted Transports
The PROXYv2 specification does not mention anything about implementing it over the encrypted transports, as it is concerned exclusively with TCP (and the so-called "dumb" TCP proxies). As such, it implies that a PROXYv2 header should be sent by the client immediately after TCP connection establishment.
Despite that, at least the dnsdist
authors decided to deviate from the spec to provide more privacy and users' data protection. That is, when PROXYv2 is enabled for TLS downstream servers, the PROXYv2 headers are sent after the handshake as opposed to sending them over TCP unencrypted, as the spec suggests.
On the other hand, HAProxy
, whose authors are largely responsible for the creation of the PROXY protocol, strictly follows the spec and always sends PROXYv2 headers ahead of any encryption. That aligns well with the fact that PROXYv2 headers should be accepted only from entrusted clients.
As far as I understand the matter, most of the software (including all possible "dumb" TCP proxies that can be used for TLS, too) will work as described in the spec (= no PRROXYv2 headers encryption). And from what I understand from the documentation, cloud infrastructure providers also follow the spec closely.
That is, we should implement both methods to remain compatible with more software and deployment scenarios. It should be noted that the dnsdist
developers also seem to have acknowledged that eventually. Starting from version 1.9.0, they provide proxyProtocolOutsideTLS
, but only for DNS over HTTPS listeners (for now?).
I would say that DNS server operators who are more interested in traditional DNS transports (aka Do53) and want to add encrypted transports to the bunch will likely gravitate towards dnsdist
, while the ones primarily interested in deploying DNS transports working on top of TLS might choose HAProxy
. The ones who rely on cloud infrastructure providers (Cloudflare, Amazon AWS, MS Azure) have no choice, and I cannot see in the documentation that their load balancers support encrypted PROXYv2 headers. Also, I think that we should follow the dnsdist
lead here and use the encrypted PROXYv2 mode by default to avoid unexpected end-user data leakage over otherwise encrypted transports.
Supporting both modes is relatively simple.
Plan for Having PROXYv2 Support in BIND (and dig)
Support for PROXYv2 in BIND echoes a lot with Stream DNS and, in general, the multilayer design approach to DNS transports. Moreover, meaningful support for PROXY would not have been possible without this.
The general idea for TCP-based transports (DNS-over-TCP, TLS, HTTPS) is creating a transport (aks PROXY Stream) that is compatible on the API level with TCP and making the multilayer transports work over it instead of TCP, similar to how Stream DNS and HTTP transports can work over either TCP and TLS. What is also very important - is that this way, the existing transports require very few changes to gain PROXYv2 support.
UDP can not be covered by this, of course, as it is datagram transport. However, we can also use the multilayer approach for datagram transports. It so happened that we did not need for this until now. Thus, we need PROXY over UDP transport, and I suspect that at some point, we will need it for QUIC, too.
Of course, we need a common code for handling PROXYv2 headers. The idea is very similar to Stream DNS: to structure the code for handling PROXYv2 headers as a state machine completely separated from networking. This way, we can test the PROXYv2 parsing code thoroughly without pumping any data via the networking stack. The code should be able to handle torn-apart PROXYv2 headers, but we can and will provide an optimised code path for the cases when we know that all data should have arrived at once (as in the case of UDP).
It is important to notice that from the configuration perspective for the end-user, it should not look as if there are any DNS-over-PROXYv2 transports. Enabling PROXYv2 is an additional mode of the existing DNS transports. Other tools with PROXYv2 support behave like this, so we should not break this expectation for users (and avoid making matters seem more complicated than they should be). Just like DNS-over-TCP and DNS-over-TLS are the same protocol implementations that differ only on the presentation layer of the OSI model, DNS transports using PROXYv2 differ from the regular ones only on the session layer (that is, even less so).
The sole purpose of PROXYv2 is to carry information about the connected client to the server for it being used instead of the connection endpoint information. On the lowest level, that means that the API that returns this information should be updated to use the information carried to the server via PROXYv2. That list includes isc_nmhandle_peeraddr()
and isc_nmhandle_localaddr()
. A similar API should be introduced for returning the real connection information (e.g. isc_nmhandle_real_peeraddr()
and isc_nmhandle_real_localaddr()
) as this information will be required in certain cases. In general, with very few low-level exceptions, most of the code should use the addresses extracted from the header. That includes all user-facing configuration options related to networking, including ACLs.
The Case of Plain and Encrypted PROXYv2 Headers over TLS-based Transports
As we have mentioned above, in the case of TLS, we should be able to deal with plain PROXYv2 headers before the TLS handshake and with encrypted ones after a TLS connection has been established. With the multilayer design, it is not a problem at all: we could swap places between TLS and PROXY Stream transports on an as-needed basis.
That is:
- When we need to use plain PROXYv2 headers, we need to run TLS Stream on top of PROXY Stream;
- When we need to use encrypted PROXYv2 headers, we need to run PROXY Stream on top of TLS Stream.
Stream DNS and HTTP/2 can work over any of the combinations: they are not concerned over what they are being used, provided that the underlying transport behaves similarly to TCP.
PROXYv2 Configuration Options in BIND
The listen-on
statement is the natural and expected configuration option to control PROXYv2 support for listeners. That also makes sense from the ease of implementation perspective. In particular, I propose to extend it with a new proxy
configuration option with the following values:
-
plain
- accept plain PROXYv2 headers. It is the only valid option for protocols that do not employ encryption. In the case of protocols that employ encryption, it instructs BIND that PROXYv2 headers are sent without encryption before the TLS handshake. In that case, only PROXYv2 headers are not encrypted. -
encrypted
- accept encrypted PROXYv2 headers. In the case of protocols that employ encryption, it instructs BIND that PROXYv2 headers are sent encrypted immediately after the TLS handshake. The option is valid only for the protocols that employ encryption.
We need encrypted
at least for compatibility with dnsdist
.
As was noted above, due to the fact that PROXYv2 allows endpoint information spoofing, we should have a way to control who is capable of sending queries with PROXYv2 headers. We cannot use the existing ACL-related options as they will work on the addresses that are extracted from the PROXYv2 headers. Thus, a new allow-proxy
and allow-proxy-on
options are required. By default, allow-proxy
should restrict PROXY queries to none
in order to have operators make decisions regarding whom to allow sending queries with PROXYv2 headers and avoid security issues related to misconfiguration. On the other hand, in order to be in line with other allow-*
options, we can set allow-proxy-on
(whose intention is to limit on which interfaces PROXY is allowed) to any
.
options {
...
# Enable PROXYv2 for Do53 (both TCP and UDP)
listen-on port 53 proxy plain { any; };
# Enable proxy for DoT, use encrypted PROXY headers for compatibility with dnsdist
listen-on port 853 proxy encrypted tls local-tls { any; };
# Enable proxy for DoH, unencryppted proxy headers for compatibility with HAProxy (and other tools)
listen-on port 443 proxy plain tls local-tls http local-http-server { any; };
# Let's allow accepting proxy only on localhost for security reasons.
# Whoever enables PROXYv2 support should explicitly specify allowed clients.
allow-proxy { 127.0.0.0/8; ::1; };
# It is also possible to specify interfaces on which PROXYv2 is allowed
allow-proxy-on { 127.0.0.1; ::1; };
...
};
Please not that proxy
is used in front of tls
and http
. That is done on purpose: the protocol options are listed in the "usual" encapsulation order.
PROXYv2-related Command Line Options in dig
With dig
- it is much simpler to add PROXYv2 support there. It can be accomplished by adding two command line arguments:
-
+[no]proxy[=src_addr[#src_port]-dst_addr[#dst_port]]
- to enable PROXYv2 support. The same option and argument format is used bykdig
, so I think it makes sense to remain compatible with it. Omitting addresses will make "dig" createLOCAL
PROXYv2 headers. -
+[no]proxy-plain
- to allow sending unencrypted PROXYv2 headers ahead of any encryption in transports working over TLS.
That should be enough to test PROXYv2 deployments in all scenarios.
Plan for PROXYv2
If we should outline the plan for implementing PROXYv2 within the BIND's codebase, it can be done as follows:
-
Implement a low-level "library" that takes care of processing incoming PROXYv2 headers as well as provides means for the creation of the headers. Implement unit tests for it. It is the most critical and foundational part. -
Implement PROXY Stream transport and use it to add support for PROXYv2 into all TCP-based transports. Reuse the existing unit tests to test both the new transport and its integration into the existing ones. -
Implement PROXY over UDP transport. Reuse UDP unit tests for the new transport. -
Add support for PROXYv2 into dig so that it can aid in the testing of the integration of PROXYv2 in BIND during the implementation phase. -
Integrate PROXYv2 support into BIND and update the documentation (ARM). -
Write system tests to verify the integration.