Skip to content

Improve reuse of outgoing TCP connections

Michał Kępień requested to merge 4693-improve-reuse-of-outgoing-tcp-connections into main

The dns_dispatch_gettcp() function is used for finding an existing TCP connection that can be reused for sending a query from a specified local address to a specified remote address. The logic for matching the provided <local address, remote address> tuple to one of the existing TCP connections is implemented in the dispatch_match() function:

  • if the examined TCP connection already has a libuv handle assigned, it means the connection has already been established; therefore, compare the provided <local address, remote address> tuple against the corresponding address tuple for the libuv handle associated with the connection,

  • if the examined TCP connection does not yet have a libuv handle assigned, it means the connection has not yet been established; therefore, compare the provided <local address, remote address> tuple against the corresponding address tuple that the TCP connection was originally created for.

This logic limits TCP connection reuse potential as the libuv handle assigned to an existing dispatch object may have a more specific local <address, port> tuple associated with it than the local <address, port> tuple that the dispatch object was originally created for. That's because the local address for outgoing connections can be set to a wildcard <address, port> tuple (indicating that the caller does not care what source <address, port> tuple will be used for establishing the connection, thereby delegating the task of picking it to the operating system) and then get "upgraded" to a specific <address, port> tuple when the socket is bound (and a libuv handle gets associated with it). When another dns_dispatch_gettcp() caller then tries to look for an existing TCP connection to the same peer and passes a wildcard address in the local part of the tuple, the function will not match that request to a previously-established TCP connection (unless isc_nmhandle_localaddr() returns a wildcard address as well).

Simplify dispatch_match() so that the libuv handle associated with an existing dispatch object is not examined for the purpose of matching it to the provided <local address, remote address> tuple; instead, always examine the <local address, remote address> tuple that the dispatch object was originally created for. This enables reuse of TCP connections created without providing a specific local socket address while still preventing other connections (created for a specific local socket address) from being inadvertently shared.

This MR is a prerequisite for !8348.

Note that dns_dispatch_gettcp() is currently only used by the dns_request API, so this MR's potential for introducing new breakage is relatively low. Furthermore, this MR will only make a practical difference once !8348 gets merged (and even then, only if the new ISC_SOCKET_DETAILS macro will be set during build). That's because isc_nmhandle_localaddr() currently simply returns handle->local and its return value will only be set to the actual address the socket is bound to with !8348 in place.

Closes #4693

Merge request reports