|
|
|
|
|
# High Level Design
|
|
|
|
|
|
This section covers the high level design of Bulk LeaseQuery (BLQ) for both DHPCv4
|
|
|
and DHCPv6, as described by [RFC 6926](https://www.rfc-editor.org/rfc/rfc6926)
|
|
|
and [RFC 5460](https://www.rfc-editor.org/rfc/rfc5460.html) respectively.
|
|
|
This section covers the high level design of Bulk LeaseQuery (BLQ) for both DHPCv4 and DHCPv6, as described by [RFC 6926](https://www.rfc-editor.org/rfc/rfc6926) and [RFC 5460](https://www.rfc-editor.org/rfc/rfc5460.html) respectively.
|
|
|
|
|
|
## Overview
|
|
|
|
|
|
Bulk LeaseQuery support will be added to the existing LeaseQuery hook
|
|
|
and anticipates future extension to support Active LeaseQuery (ALQ). There
|
|
|
are some aspects of Basic LeaseQuery implementation that can be reused, though
|
|
|
some refactoring of that existing code is likely.
|
|
|
|
|
|
Running Bulk LeaseQuery will require Kea to be configured in MT mode. BLQ
|
|
|
would have significant performance impacts on single-threaded configurations
|
|
|
and thus it will not be supported. This should simplify the design as well
|
|
|
as the implementation. The LeaseQuery hook library will verify the MT is
|
|
|
enabled if BLQ is enabled.
|
|
|
|
|
|
BLQ will support a configurable number of concurrent TCP connections. Each
|
|
|
connection belongs to a single requester who may request multiple queries,
|
|
|
each with a XID unique to the connection. A requester need not wait for one
|
|
|
query to complete before asking for another. In fact, a single inbound
|
|
|
requester packet can contain multiple queries.
|
|
|
|
|
|
The RFCs state that servers may process a requester's queries in parallel and
|
|
|
intermingle the binding responses sent back, whether the queries arrive in a
|
|
|
single request packet or multiple packets. This design provides support for a
|
|
|
configurable number of parallel queries per requester.
|
|
|
|
|
|
There is no inherent difference between a connection for Bulk queries versus
|
|
|
a connection for Active queries, other than the requester sending an optional
|
|
|
DHCPTLS message and then a DHCPACTIVELEASEQUERY. In theory, a Bulk requester
|
|
|
could morph their connection into an Active connection, simply by sending
|
|
|
DHCPACTIVELEASEQUERY. While this design does not add anything to directly
|
|
|
support ALQ, rather it lays the ground to work add it in the future.
|
|
|
Bulk LeaseQuery support will be added to the existing LeaseQuery hook and anticipates future extension to support Active LeaseQuery (ALQ). There are some aspects of Basic LeaseQuery implementation that can be reused, though some refactoring of that existing code is likely.
|
|
|
|
|
|
Running Bulk LeaseQuery will require Kea to be configured in MT mode. BLQ would have significant performance impacts on single-threaded configurations and thus it will not be supported. This should simplify the design as well as the implementation. The LeaseQuery hook library will verify the MT is enabled if BLQ is enabled.
|
|
|
|
|
|
BLQ will support a configurable number of concurrent TCP connections. Each connection belongs to a single requester who may request multiple queries, each with a XID unique to the connection. A requester need not wait for one query to complete before asking for another. In fact, a single inbound requester packet can contain multiple queries.
|
|
|
|
|
|
The RFCs state that servers may process a requester's queries in parallel and intermingle the binding responses sent back, whether the queries arrive in a single request packet or multiple packets. This design provides support for a configurable number of parallel queries per requester.
|
|
|
|
|
|
There is no inherent difference between a connection for Bulk queries versus a connection for Active queries, other than the requester sending an optional DHCPTLS message and then a DHCPACTIVELEASEQUERY. In theory, a Bulk requester could morph their connection into an Active connection, simply by sending DHCPACTIVELEASEQUERY. While this design does not add anything to directly support ALQ, rather it lays the ground to work add it in the future.
|
|
|
|
|
|
The design is described in the following sections:
|
|
|
|
... | ... | @@ -48,29 +26,21 @@ The design is described in the following sections: |
|
|
|
|
|
## Process Flow
|
|
|
|
|
|
Unlike normal DHCP or our current REST API traffic, Bulk LeaseQuery is not a typical request/response
|
|
|
exchange. Each inbound requester packet can contain multiple queries. The outbound server response
|
|
|
can contain multiple replies for multiple queries, intermixed, and can span multiple packets.
|
|
|
Unlike normal DHCP or our current REST API traffic, Bulk LeaseQuery is not a typical request/response exchange. Each inbound requester packet can contain multiple queries. The outbound server response can contain multiple replies for multiple queries, intermixed, and can span multiple packets.
|
|
|
|
|
|
This design will treat inbound data as a stream of DHCP query requests, each with a unique XID, and where individual TCP packets are simple freight containers conveying them rather than the request half of a transaction. The same perspective will applied to the outbound responses, they will be a stream of bindings and status responses (e.g. LEASEQUERYDONE) each tagged with corresponding XID.
|
|
|
This design will treat inbound data as a stream of DHCP query requests, each with a unique XID, and where individual TCP packets are simple freight containers conveying them rather than the request half of a transaction. The same perspective will applied to the outbound responses, they will be a stream of bindings and status responses (e.g. LEASEQUERYDONE) each tagged with corresponding XID.
|
|
|
|
|
|
The design introduces a new class, LeaseQueryListener, which will support multiple, concurrent
|
|
|
TCP connections, one connection per requester, one thread per connection. It will be very similar to and reuse much of the class hierarchy as CmdHttpListener, including TLS support. While a good deal the existing class hierarchy topped by CmdHttpListener can be used, at least as a model for the higher order functions, the HttpConnection::Transaction and the logic using it is not quite the right fit. It is intended to read a single HTTP request that generates a single HTTP response,though in theory, either can span multiple packets. The requisite functions will need to modified accordingly (e.g. doRead(), socketReadCallback()).
|
|
|
The design introduces a new class, LeaseQueryListener, which will support multiple, concurrent TCP connections, one connection per requester, one thread per connection. It will be very similar to and reuse much of the class hierarchy as CmdHttpListener, including TLS support. While a good deal the existing class hierarchy topped by CmdHttpListener can be used, at least as a model for the higher order functions, the HttpConnection::Transaction and the logic using it is not quite the right fit. It is intended to read a single HTTP request that generates a single HTTP response,though in theory, either can span multiple packets. The requisite functions will need to modified accordingly (e.g. doRead(), socketReadCallback()).
|
|
|
|
|
|
We can follow the pattern used by LeaseMgr::LeaseStatsQuery to create BulkLeaseQuery class. This class
|
|
|
will will be used for starting a new bulk query and fetching its results. Each lease store will implement
|
|
|
its own derivations, and in this way, isolate lease store specifics.
|
|
|
We can follow the pattern used by LeaseMgr::LeaseStatsQuery to create BulkLeaseQuery class. This class will will be used for starting a new bulk query and fetching its results. Each lease store will implement its own derivations, and in this way, isolate lease store specifics.
|
|
|
|
|
|
LeaseQueryListener will provide a connection thread pool in which each thread supports a single
|
|
|
connection to one requester. Each connection thread will be responsible for reading incoming query
|
|
|
packets and sending outgoing response packets.
|
|
|
LeaseQueryListener will provide a connection thread pool in which each thread supports a single connection to one requester. Each connection thread will be responsible for reading incoming query packets and sending outgoing response packets.
|
|
|
|
|
|
The following diagram illustrates the new class hierarchy:
|
|
|
|
|
|
![blq_classes.svg](uploads/84777a6a72d6a946a6950922d2b08c58/blq_classes.svg)
|
|
|
|
|
|
When a query packet arrives its queries will be pushed onto a queue. The queue is monitored a pool
|
|
|
by one or more query worker threads. Each worker thread is responsible for submitting a single query against the lease store, iteratively fetching the results, and pushing results as XID/binding pairs to connection's outbound queue. The connection thread will add the binding pairs to the current TCP packet until it is full and then send it asynchronously. Subsequent pairs are adding to a new packet. This continues until there are no more queued queries.
|
|
|
When a query packet arrives its queries will be pushed onto a queue. The queue is monitored a pool by one or more query worker threads. Each worker thread is responsible for submitting a single query against the lease store, iteratively fetching the results, and pushing results as XID/binding pairs to connection's outbound queue. The connection thread will add the binding pairs to the current TCP packet until it is full and then send it asynchronously. Subsequent pairs are adding to a new packet. This continues until there are no more queued queries.
|
|
|
|
|
|
A conceptualization of the processing in pseudo code follows:
|
|
|
|
... | ... | @@ -123,66 +93,57 @@ TcpConnection::pushToSend(xid, binding) { |
|
|
}
|
|
|
```
|
|
|
|
|
|
We may need a gating mechanism (e.g. condition variable in pushToSend()), based on
|
|
|
a maximum number of bindings queued for send (per connection) so query workers do not
|
|
|
create enormous outbound queues. In other words, we need to avoid outrunning the
|
|
|
connection's ability to send by too avoid memory exhausting et al. In practice this
|
|
|
may be a non-issue but it is better to allow for it now.
|
|
|
We may need a gating mechanism (e.g. condition variable in pushToSend()), based on a maximum number of bindings queued for send (per connection) so query workers do not create enormous outbound queues. In other words, we need to avoid outrunning the connection's ability to send by too avoid memory exhausting et al. In practice this may be a non-issue but it is better to allow for it now.
|
|
|
|
|
|
## Query Execution
|
|
|
|
|
|
It is possible for queries to select very large numbers of leases, certainly
|
|
|
far more than could be returned in a single fetch. While this is not a necessarily
|
|
|
a problem for Memfile lease storage, it represents a significant consideration
|
|
|
for DB lease storage. This can be approached at least two ways: use cursors or use
|
|
|
iterative queries.
|
|
|
It is possible for queries to select very large numbers of leases, certainly far more than could be returned in a single fetch. While this is not a necessarily a problem for Memfile lease storage, it represents a significant consideration for DB lease storage. This can be approached at least two ways: use cursors or use iterative queries.
|
|
|
|
|
|
### Cursors
|
|
|
|
|
|
With cursors the desired query is submitted to the DB once, and the results
|
|
|
are fetched piece meal. While we have not used this approach within Kea yet,
|
|
|
it maybe it does place the burden or performance on the DB server.
|
|
|
With cursors the desired query is submitted to the DB once, and the results are fetched piece meal. While we have not used this approach within Kea yet, it maybe it does place the burden or performance on the DB server.
|
|
|
|
|
|
#### Postgresql
|
|
|
**Postgresql**
|
|
|
|
|
|
Cursors are identified by name in Posgresql, names need only be unique
|
|
|
within a session. The cursor must first be created for the given SQL
|
|
|
statement, and then used in iterative fetch invocations to retrieve
|
|
|
the query results by fetching the NEXT n records:
|
|
|
```
|
|
|
Cursors are identified by name in Posgresql, names need only be unique
|
|
|
within a session. The cursor must first be created for the given SQL
|
|
|
statement, and then used in iterative fetch invocations to retrieve
|
|
|
the query results by fetching the NEXT n records:
|
|
|
```
|
|
|
|
|
|
```
|
|
|
FETCH NEXT n FROM cursor INTO rowvar
|
|
|
```
|
|
|
as illustrated here:
|
|
|
|
|
|
https://www.postgresql.org/docs/current/plpgsql-cursors.html
|
|
|
```
|
|
|
as illustrated here:
|
|
|
|
|
|
(SIDEBAR - cursors might come in handy for use with expiration some day)
|
|
|
https://www.postgresql.org/docs/current/plpgsql-cursors.html
|
|
|
|
|
|
#### MySQL
|
|
|
(SIDEBAR - cursors might come in handy for use with expiration some day)
|
|
|
```
|
|
|
|
|
|
**MySQL**
|
|
|
|
|
|
To use cursors with a prepared statement we need to set requisite statement attributes
|
|
|
with mysq_stmt_attr_set(). Note, that one of those parameters, STMT_ATTR_PREFETCH_ROWS,
|
|
|
determines the number of rows returned on each fetch:
|
|
|
```
|
|
|
To use cursors with a prepared statement we need to set requisite statement attributes
|
|
|
with mysq_stmt_attr_set(). Note, that one of those parameters, STMT_ATTR_PREFETCH_ROWS,
|
|
|
determines the number of rows returned on each fetch:
|
|
|
|
|
|
https://dev.mysql.com/doc/c-api/5.6/en/mysql-stmt-attr-set.html
|
|
|
https://dev.mysql.com/doc/c-api/5.6/en/mysql-stmt-attr-set.html
|
|
|
|
|
|
Statement ID serves as cursor identifier. Do not call mysql_stmt_store_result(),
|
|
|
but make repeated calls to mysql_stmt_fetch(). I THINK that behind the scenes
|
|
|
mysql client lib returns to server when you exhaust the local fetched records,
|
|
|
rather then for every row.
|
|
|
Statement ID serves as cursor identifier. Do not call mysql_stmt_store_result(),
|
|
|
but make repeated calls to mysql_stmt_fetch(). I THINK that behind the scenes
|
|
|
mysql client lib returns to server when you exhaust the local fetched records,
|
|
|
rather then for every row.
|
|
|
```
|
|
|
|
|
|
### Iterative queries
|
|
|
|
|
|
The queries would order by ip-address and apply either a starting address or
|
|
|
a numerical offset and a row limit. When the current fetched results has been consumed,
|
|
|
a subsequent query is issued asking for rows with addresses > then the maximum address of the
|
|
|
previous fetch. The initial query would use a beginning address of 0.
|
|
|
The queries would order by ip-address and apply either a starting address or a numerical offset and a row limit. When the current fetched results has been consumed, a subsequent query is issued asking for rows with addresses > then the maximum address of the previous fetch. The initial query would use a beginning address of 0.
|
|
|
|
|
|
This is akin to doing our own cursors but differs in performance portent, they may be
|
|
|
less efficient. Each call is a new SQL execution, in addition to needing to sort the
|
|
|
results by IP address (though we have existing index for this). Example selects for
|
|
|
either method for PostgreSQL are shown below (MySQL would be similar):
|
|
|
This is akin to doing our own cursors but differs in performance portent, they may be less efficient. Each call is a new SQL execution, in addition to needing to sort the results by IP address (though we have existing index for this). Example selects for either method for PostgreSQL are shown below (MySQL would be similar):
|
|
|
|
|
|
Using a starting IP address ("$1" is always the last IP address returned from the prior query):
|
|
|
|
... | ... | @@ -197,9 +158,7 @@ ORDER by ip-addr |
|
|
LIMIT $3
|
|
|
```
|
|
|
|
|
|
Or per this page https://www.postgresql.org/docs/12/queries-limit.html, you can
|
|
|
use skip the first n rows by using OFFSET and LIMIT together with ORDER BY:
|
|
|
|
|
|
Or per this page https://www.postgresql.org/docs/12/queries-limit.html, you can use skip the first n rows by using OFFSET and LIMIT together with ORDER BY:
|
|
|
|
|
|
```
|
|
|
SELECT *
|
... | ... | @@ -210,25 +169,19 @@ ORDER by ip-addr |
|
|
LIMIT $2
|
|
|
OFFSET $3
|
|
|
```
|
|
|
|
|
|
where "$3" is the total number of rows already returned.
|
|
|
|
|
|
This does imply that subsequent query invocations would repeat some portion of the
|
|
|
record selection of previous queries. Consider moving through 100K leases 10K at a time.
|
|
|
You have to skip 10K, then 20K, then 30K records and then start fetching against
|
|
|
the LIMIT. Most DB servers do retain some amount of caching per session, so this may be
|
|
|
something of a non-issue
|
|
|
This does imply that subsequent query invocations would repeat some portion of the record selection of previous queries. Consider moving through 100K leases 10K at a time. You have to skip 10K, then 20K, then 30K records and then start fetching against the LIMIT. Most DB servers do retain some amount of caching per session, so this may be something of a non-issue
|
|
|
|
|
|
On the positive side, using this technique is something we already do with the
|
|
|
leaseX-get-page commands in lease_cmds hook library. Additionally, the SELECT and
|
|
|
results fetch are effectively one operation, perhaps reducing the chance for
|
|
|
concurrent lease changes.
|
|
|
On the positive side, using this technique is something we already do with the leaseX-get-page commands in lease_cmds hook library. Additionally, the SELECT and results fetch are effectively one operation, perhaps reducing the chance for concurrent lease changes.
|
|
|
|
|
|
## Query Data Requirements
|
|
|
|
|
|
Looking more specifically at the query data requirements, a list of the queries called
|
|
|
for by the BLQ RFCs follows:
|
|
|
Looking more specifically at the query data requirements, a list of the queries called for by the BLQ RFCs follows:
|
|
|
|
|
|
**V4 Queries**
|
|
|
|
|
|
V4 Queries:
|
|
|
1. By MAC
|
|
|
2. By Client ID
|
|
|
3. By Relay ID (client supplies via RAI sub option 12)
|
... | ... | @@ -236,18 +189,15 @@ V4 Queries: |
|
|
5. By VPN ID (client supplies via VPN select option, see RFC 6607) - if we choose to support it
|
|
|
6. All Configured IP Addresses
|
|
|
|
|
|
V6 Queries:
|
|
|
1. Relay ID (option 53 D60_RELAY_ID extracted from Pkt6::relay_info_.options_)
|
|
|
**V6 Queries**
|
|
|
|
|
|
1. Relay ID (option 53 D60_RELAY_ID extracted from Pkt6::relay_info\_.options\_)
|
|
|
2. Link address - from existing extended relay-info
|
|
|
3. Remote ID (option 37 D60_REMOTE_ID extracted from RelayInfo)
|
|
|
|
|
|
In order to query for leases by these parameters, their values need to be readily accessible
|
|
|
for each lease within the lease store.
|
|
|
In order to query for leases by these parameters, their values need to be readily accessible for each lease within the lease store.
|
|
|
|
|
|
When Basic LeaseQuery was implemented, it added a global configuration
|
|
|
parameter, "store-extended-info", which enables Kea core code to store
|
|
|
relay information for each lease. For V4 this consists of the RAI option (82)
|
|
|
stored as a hex literal:
|
|
|
When Basic LeaseQuery was implemented, it added a global configuration parameter, "store-extended-info", which enables Kea core code to store relay information for each lease. For V4 this consists of the RAI option (82) stored as a hex literal:
|
|
|
|
|
|
```
|
|
|
"ISC":
|
... | ... | @@ -274,9 +224,7 @@ and for V6, we store an entry for each relay layer in an array: |
|
|
}
|
|
|
```
|
|
|
|
|
|
Querying for leases based on option values that are embedded in the packed data
|
|
|
stored as "options" is untenable. Therefore, the design proposes adding individual
|
|
|
values for each parameter to the user-context.
|
|
|
Querying for leases based on option values that are embedded in the packed data stored as "options" is untenable. Therefore, the design proposes adding individual values for each parameter to the user-context.
|
|
|
|
|
|
For V4, the context would appear as follows:
|
|
|
|
... | ... | @@ -288,7 +236,6 @@ For V4, the context would appear as follows: |
|
|
"relay-id" : 0x... // extracted from RAI
|
|
|
...
|
|
|
}
|
|
|
|
|
|
```
|
|
|
|
|
|
and for v6:
|
... | ... | @@ -311,21 +258,12 @@ and for v6: |
|
|
}
|
|
|
```
|
|
|
|
|
|
These new values could be populated in the lease's user_context in at least two
|
|
|
ways:
|
|
|
These new values could be populated in the lease's user_context in at least two ways:
|
|
|
|
|
|
1. Done in Kea core in AllocEngine::updateLease<4/6>ExtendedInfo() functions.
|
|
|
These two functions only store lease information in the user-context when
|
|
|
the "store-extended-info" parameter is true. We would simply be adding a bit
|
|
|
more code here.
|
|
|
1. Done in Kea core in AllocEngine::updateLease<4/6>ExtendedInfo() functions. These two functions only store lease information in the user-context when the "store-extended-info" parameter is true. We would simply be adding a bit more code here.
|
|
|
2. Within call outs for hook points such as lease<4/6>\_select, lease<4/6>\_renew that BLQ would implement. These call outs could also subsume the code now done n updateLease<4/6>ExtendedInfo() in addition to adding the new explicit values.
|
|
|
|
|
|
2. Within call outs for hook points such as lease<4/6>_select, lease<4/6>_renew
|
|
|
that BLQ would implement. These call outs could also subsume the code now done n
|
|
|
updateLease<4/6>ExtendedInfo() in addition to adding the new explicit values.
|
|
|
|
|
|
At this point, regardless of the lease store used, the LeaseX::user_context_ would
|
|
|
contain individual JSON key/value pairs for all of the BLQ query parameters. However,
|
|
|
this alone is not enough to allow for efficient querying.
|
|
|
At this point, regardless of the lease store used, the LeaseX::user_context\_ would contain individual JSON key/value pairs for all of the BLQ query parameters. However, this alone is not enough to allow for efficient querying.
|
|
|
|
|
|
### V4 Query time parameters, STOS, and CLTT
|
|
|
|
... | ... | @@ -353,15 +291,9 @@ and also Sec 8.3, item 3: |
|
|
dhcp-state option's state became valid.
|
|
|
```
|
|
|
|
|
|
Kea does not store state-time-of-state (STOS) as does ISC DHCP and CLTT is derived by
|
|
|
calculation: expiration - valid_lft. In other words, currently Kea has a derived hybrid of
|
|
|
STOS/CLTT. First, we would probably need to document that Kea does not currently track
|
|
|
STOS, and as such only base query results on our notion of CLTT.
|
|
|
Kea does not store state-time-of-state (STOS) as does ISC DHCP and CLTT is derived by calculation: expiration - valid_lft. In other words, currently Kea has a derived hybrid of STOS/CLTT. First, we would probably need to document that Kea does not currently track STOS, and as such only base query results on our notion of CLTT.
|
|
|
|
|
|
Second, in order to avoid full table scans, we would likely need an index on a
|
|
|
generated column for the calculation. We will need a function to generate such a
|
|
|
column, as we must account for infinite lifetime leases, as there is a bit of
|
|
|
trickery done when such a lease is stored:
|
|
|
Second, in order to avoid full table scans, we would likely need an index on a generated column for the calculation. We will need a function to generate such a column, as we must account for infinite lifetime leases, as there is a bit of trickery done when such a lease is stored:
|
|
|
|
|
|
```
|
|
|
// The lease structure holds the client last transmission time (cltt_)
|
... | ... | @@ -378,7 +310,7 @@ trickery done when such a lease is stored: |
|
|
}
|
|
|
```
|
|
|
|
|
|
Thus the stored value for expire_ is already CLTT.
|
|
|
Thus the stored value for expire\_ is already CLTT.
|
|
|
|
|
|
### V4 Query for All Configured IP Addresses
|
|
|
|
... | ... | @@ -390,6 +322,7 @@ RFC6929/Sec 7.2: |
|
|
A Query for All Configured IP addresses is signaled by the absence
|
|
|
of any other primary query.
|
|
|
```
|
|
|
|
|
|
and section 8.2:
|
|
|
|
|
|
```
|
... | ... | @@ -399,9 +332,7 @@ and section 8.2: |
|
|
binding to a DHCPv4 client MUST be returned in a DHCPLEASEUNASSIGNED message.
|
|
|
```
|
|
|
|
|
|
Thus kea-dhcp4 will need to extrapolate all of the free leases based on pool
|
|
|
knowledge. Remembering that assigned leases are fetched, ordered by lease address,
|
|
|
then the algorithm to process the results might look something like this:
|
|
|
Thus kea-dhcp4 will need to extrapolate all of the free leases based on pool knowledge. Remembering that assigned leases are fetched, ordered by lease address, then the algorithm to process the results might look something like this:
|
|
|
|
|
|
```
|
|
|
current_pool = none
|
... | ... | @@ -458,54 +389,35 @@ if (last_sent < pool_end) { |
|
|
pushToSend(XID, free_lease);
|
|
|
}
|
|
|
}
|
|
|
|
|
|
```
|
|
|
|
|
|
There is also some questions as to how to treat host reservations for fixed addresses.
|
|
|
Those that are in-pool but without a lease would be sent back as free by extrapolation.
|
|
|
Technically accurate in terms of lease state but functionally it is only available to
|
|
|
a single client. For out-of-pool addresses without leases, we would send back nothing
|
|
|
without doing some sort of explicit search for them.
|
|
|
There is also some questions as to how to treat host reservations for fixed addresses. Those that are in-pool but without a lease would be sent back as free by extrapolation. Technically accurate in terms of lease state but functionally it is only available to a single client. For out-of-pool addresses without leases, we would send back nothing without doing some sort of explicit search for them.
|
|
|
|
|
|
In order to avoid potentially costly searches against host reservations, it is proposed
|
|
|
that we simply document the above as the expected behavior.
|
|
|
In order to avoid potentially costly searches against host reservations, it is proposed that we simply document the above as the expected behavior.
|
|
|
|
|
|
### V4 Query by VPN ID
|
|
|
|
|
|
As has been suggested by @fdupont: "VPNs are already not supported by Kea, e.g. you can't define multiple subnets using 10.0.0.0/8 private range saying they are in different VPNs (note we can do this one day: I simply believe we wait for an explicit request from a customer...)".
|
|
|
As has been suggested by @fdupont: "VPNs are already not supported by Kea, e.g. you can't define multiple subnets using 10.0.0.0/8 private range saying they are in different VPNs (note we can do this one day: I simply believe we wait for an explicit request from a customer...)".\
|
|
|
There is little value in trying to support it in BLQ at this juncture.
|
|
|
|
|
|
## Query Mechanics
|
|
|
|
|
|
Initially, it looked as though PostgreSQL and MySQL both supported sufficient JSON
|
|
|
mechanisms to index and query by JSON. However, further exploration revealed the
|
|
|
following:
|
|
|
|
|
|
1. PostgreSQL does support Generalized Inverted (GIN) indexes on JSONB columns, and
|
|
|
those mechanisms appear sufficient to address both V4 and V6 needs. However,
|
|
|
JSONB columns are not supported until PostgreSQL 9.4. In order to use GIN indexes
|
|
|
the user_context column type in the leaseX tables would have to be changed from TEXT
|
|
|
to JSONB. This would necessitate changing many of the existing SQL statements in
|
|
|
postgresql lease manager. Furthermore, it would mean Kea's PostgreSQL
|
|
|
schema could no longer with PostgreSQL pre 9.4.
|
|
|
Initially, it looked as though PostgreSQL and MySQL both supported sufficient JSON mechanisms to index and query by JSON. However, further exploration revealed the following:
|
|
|
|
|
|
2. MySQL does not support indexing on JSON columns at all, in any version. Nor do
|
|
|
they seem likely to do so. For indexed queries (i.e not full table scans) would
|
|
|
require using generated columns, explicit columns, or a cross-reference table(s).
|
|
|
1. PostgreSQL does support Generalized Inverted (GIN) indexes on JSONB columns, and those mechanisms appear sufficient to address both V4 and V6 needs. However, JSONB columns are not supported until PostgreSQL 9.4. In order to use GIN indexes the user_context column type in the leaseX tables would have to be changed from TEXT to JSONB. This would necessitate changing many of the existing SQL statements in postgresql lease manager. Furthermore, it would mean Kea's PostgreSQL schema could no longer with PostgreSQL pre 9.4.
|
|
|
2. MySQL does not support indexing on JSON columns at all, in any version. Nor do they seem likely to do so. For indexed queries (i.e not full table scans) would require using generated columns, explicit columns, or a cross-reference table(s).
|
|
|
|
|
|
### V4 solutions
|
|
|
|
|
|
#### Memfile
|
|
|
Lease::user_context_ is an ElementPtrUsing so it should be possible to create predicate
|
|
|
objects to extract the query value to create new indexes for the lease multi-index-container
|
|
|
such as is described here:
|
|
|
**Memfile**
|
|
|
|
|
|
Lease::user_context\_ is an ElementPtrUsing so it should be possible to create predicate objects to extract the query value to create new indexes for the lease multi-index-container such as is described here:
|
|
|
|
|
|
https://stackoverflow.com/questions/1628321/boost-multi-index-container-with-index-based-on-nested-values
|
|
|
|
|
|
#### MySQL
|
|
|
**MySQL**
|
|
|
|
|
|
Use a generated column for each query value. This would require either a function or
|
|
|
an expression to extract the value from user_context, similar to this:
|
|
|
Use a generated column for each query value. This would require either a function or an expression to extract the value from user_context, similar to this:
|
|
|
|
|
|
```
|
|
|
gen_relay_id varbinary(128) GENERATED ALWAYS AS (getRelayIdFromUserContext(user_context)),
|
... | ... | @@ -515,14 +427,11 @@ an expression to extract the value from user_context, similar to this: |
|
|
...
|
|
|
```
|
|
|
|
|
|
Generated columns are automatically calculated on record INSERT and UPDATE, without the extra
|
|
|
code involved with adding new explicit columns that are populated by triggers.
|
|
|
Generated columns are automatically calculated on record INSERT and UPDATE, without the extra code involved with adding new explicit columns that are populated by triggers.
|
|
|
|
|
|
#### PostgreSQL:
|
|
|
**PostgreSQL**
|
|
|
|
|
|
Assuming we choose to use JSONB/GIN some examples are select statements are worth exploring.
|
|
|
These statements include data that is similar in structure to V6 user-context content and uses
|
|
|
a JSONB boolean expression similar to what we would employ for BLQ. The followings statement returns true (t):
|
|
|
Assuming we choose to use JSONB/GIN some examples are select statements are worth exploring. These statements include data that is similar in structure to V6 user-context content and uses a JSONB boolean expression similar to what we would employ for BLQ. The followings statement returns true (t):
|
|
|
|
|
|
```
|
|
|
select '{ "relay-info":[ { "remote-id":10, "hop":1 }, { "remote-id":20, "hop":2 }]}'::jsonb @> '{"relay-info":[{"remote-id":20}]}'::jsonb;
|
... | ... | @@ -536,8 +445,7 @@ select '{ "relay-info":[ { "remote-id":10, "hop":1 }, { "remote-id":20, "hop":2 |
|
|
select '{ "no-relay-info": "nada" }'::jsonb @> '{"relay-info":[{"remote-id":20}]}'::jsonb;
|
|
|
```
|
|
|
|
|
|
The user_context column for both lease4 and lease6 would need to be changed to JSONB and we will need
|
|
|
to add GIN (Generalized Inverted Index) indexes for them:
|
|
|
The user_context column for both lease4 and lease6 would need to be changed to JSONB and we will need to add GIN (Generalized Inverted Index) indexes for them:
|
|
|
|
|
|
```
|
|
|
ALTER TABLE lease4 ALTER COLUMN user_context TYPE JSONB using user_context::jsonb;
|
... | ... | @@ -546,22 +454,17 @@ ALTER TABLE lease6 ALTER COLUMN user_context TYPE JSONB using user_context::json |
|
|
CREATE INDEX lease6_user_context_gin ON lease6 USING GIN (user_context);
|
|
|
```
|
|
|
|
|
|
Note it will be necessary to do more robust performance testing to formulate the queries and the GIN
|
|
|
indexes. The above code is demonstrative. PostgreSQL GIN index expressions are fairly rich and
|
|
|
we should be able to tailor them to meet differing needs between V4 and V6 to maximize performance.
|
|
|
Note it will be necessary to do more robust performance testing to formulate the queries and the GIN indexes. The above code is demonstrative. PostgreSQL GIN index expressions are fairly rich and we should be able to tailor them to meet differing needs between V4 and V6 to maximize performance.
|
|
|
|
|
|
If we choose not to use JSONB/GIN, we can use the generated column approach here too.
|
|
|
PostgreSQL supports it and it would avoid incompatibility with older PostgreSQL versions.
|
|
|
If we choose not to use JSONB/GIN, we can use the generated column approach here too. PostgreSQL supports it and it would avoid incompatibility with older PostgreSQL versions.
|
|
|
|
|
|
### V6 solutions
|
|
|
|
|
|
V6 adds a many-to-many wrinkle to the mix: Each lease could pass through multiple relay layers,
|
|
|
each of which has its own link address and may add relay-id, remote-id or both (among other options).
|
|
|
V6 adds a many-to-many wrinkle to the mix: Each lease could pass through multiple relay layers, each of which has its own link address and may add relay-id, remote-id or both (among other options).
|
|
|
|
|
|
#### MySQL
|
|
|
**MySQL**
|
|
|
|
|
|
For MySQL the only reasonable way I can see is to create a cross-reference table that is maintained
|
|
|
by triggers on create, update, and delete lease. The table might look something like this:
|
|
|
For MySQL the only reasonable way I can see is to create a cross-reference table that is maintained by triggers on create, update, and delete lease. The table might look something like this:
|
|
|
|
|
|
```
|
|
|
CREATE TABLE IF NOT EXISTS lease6_relay {
|
... | ... | @@ -580,35 +483,19 @@ CREATE TABLE IF NOT EXISTS lease6_relay { |
|
|
};
|
|
|
```
|
|
|
|
|
|
#### PostgreSQL
|
|
|
**PostgreSQL**
|
|
|
|
|
|
If we choose not to employ JSONB/GIN, the we would need to create a similar cross-reference
|
|
|
table as done for MySQL.
|
|
|
If we choose not to employ JSONB/GIN, the we would need to create a similar cross-reference table as done for MySQL.
|
|
|
|
|
|
For either DB, it would be relatively difficult to populate the cross-reference table from
|
|
|
existing lease data during an upgrade. The existing leases' user-contexts would have to include
|
|
|
extended information stored from which to extract the values. Parsing those value in shell
|
|
|
or SQL would be rather ugly (though possible). It would likely be best to treat this as we
|
|
|
lease limits, and declare that it will only work for leases assigned or renewed after BLQ has been
|
|
|
installed.
|
|
|
For either DB, it would be relatively difficult to populate the cross-reference table from existing lease data during an upgrade. The existing leases' user-contexts would have to include extended information stored from which to extract the values. Parsing those value in shell or SQL would be rather ugly (though possible). It would likely be best to treat this as we lease limits, and declare that it will only work for leases assigned or renewed after BLQ has been installed.
|
|
|
|
|
|
#### Memfile
|
|
|
**Memfile**
|
|
|
|
|
|
For Memfile_LeaseMgr, we would mimic the cross-reference table with a new multi_index_container
|
|
|
and maintain it by adding calls to maintenance functions inside addLease(), updateLease(), and
|
|
|
deleteLease(), similar to what was done to support lease limits. The function calls could be
|
|
|
skipped if store-extended-info is false. The cross-reference container would need to be initialized
|
|
|
during dhcp_srv_configured callback, as loading Memfile stores leases directly into the container,
|
|
|
rather than calling addLease(). This would only occur when BLQ is enabled. Note the container
|
|
|
contents are not persisted across reboots.
|
|
|
For Memfile_LeaseMgr, we would mimic the cross-reference table with a new multi_index_container and maintain it by adding calls to maintenance functions inside addLease(), updateLease(), and deleteLease(), similar to what was done to support lease limits. The function calls could be skipped if store-extended-info is false. The cross-reference container would need to be initialized during dhcp_srv_configured callback, as loading Memfile stores leases directly into the container, rather than calling addLease(). This would only occur when BLQ is enabled. Note the container contents are not persisted across reboots.
|
|
|
|
|
|
## Configuration
|
|
|
|
|
|
Current Lease Query hook library config has a single parameter, ``requesters`` which is
|
|
|
a list of addresses from whom Basic LeaseQuery messages are accepted. Bulk LeaseQuery
|
|
|
will require some additions. Because Active LeaseQuery builds upon Bulk LeaseQuery,
|
|
|
the additions have been lumped into a new element called "advanced" and is depicted
|
|
|
below:
|
|
|
Current Lease Query hook library config has a single parameter, `requesters` which is a list of addresses from whom Basic LeaseQuery messages are accepted. Bulk LeaseQuery will require some additions. Because Active LeaseQuery builds upon Bulk LeaseQuery, the additions have been lumped into a new element called "advanced" and is depicted below:
|
|
|
|
|
|
```
|
|
|
"hooks-libraries": [
|
... | ... | @@ -639,27 +526,15 @@ below: |
|
|
],
|
|
|
```
|
|
|
|
|
|
* "bulk-query-enabled" - when true, Kea will accept connections from IPs in the requesters list
|
|
|
and honor process BLQ queries received
|
|
|
|
|
|
* "active-query-enabled" - when true, Kea will accept connections from IPs in the requesters
|
|
|
list and allow establishment of active query sessions. Anticipated parameter,
|
|
|
ALQ is not YET covered by this design.
|
|
|
|
|
|
* "bulk-query-enabled" - when true, Kea will accept connections from IPs in the requesters list and honor process BLQ queries received
|
|
|
* "active-query-enabled" - when true, Kea will accept connections from IPs in the requesters list and allow establishment of active query sessions. Anticipated parameter, ALQ is not YET covered by this design.
|
|
|
* "lease-query-ip" - ip address upon which to listen for advanced query connections
|
|
|
* "lease-query-port - port upon which to listen, defaults to 67 for v4 and 547 for v6
|
|
|
|
|
|
* "max-requester-connections" - maximum number of concurrent requester connections (must be > 0)
|
|
|
|
|
|
* "max-concurrent-queries" - maximum number of concurrent queries per connection. A value 0 leaves
|
|
|
the number for Kea to determine.
|
|
|
|
|
|
* "max-requester-idle-time" - amount time that may elapse between receiving data from a requester
|
|
|
before its connection is closed as idle.
|
|
|
|
|
|
* "max-concurrent-queries" - maximum number of concurrent queries per connection. A value 0 leaves the number for Kea to determine.
|
|
|
* "max-requester-idle-time" - amount time that may elapse between receiving data from a requester before its connection is closed as idle.
|
|
|
* "max-send-queue-size" - maximum number of outbound bindings queued to send (per connection)
|
|
|
|
|
|
* "max-leases-per-fetch" - maximum number of leases to return in a single fetch
|
|
|
* "max-leases-per-fetch" - maximum number of leases to return in a single fetch
|
|
|
|
|
|
The TLS parameters are self-explanatory.
|
|
|
|
... | ... | @@ -667,31 +542,20 @@ The TLS parameters are self-explanatory. |
|
|
|
|
|
### dhcp-state "available" vs "remote"
|
|
|
|
|
|
RFC 6926/Sec 6.2.7, talks about lease states of "available" or "remote" when they
|
|
|
are unassigned based upon whether or not they are currently in scope for (i.e. held
|
|
|
by) the server responding to the LeaseQuery.
|
|
|
RFC 6926/Sec 6.2.7, talks about lease states of "available" or "remote" when they are unassigned based upon whether or not they are currently in scope for (i.e. held by) the server responding to the LeaseQuery.
|
|
|
|
|
|
Unlike ISC DHCP, Kea does track which server in an HA configuration currently holds
|
|
|
a free lease, nor is there an efficient way for a server to know if such a lease is
|
|
|
in the server's scope, outside of the HA hook library. As this differentiation
|
|
|
in state is not required by the RFC, Kea will not be implementing it as this time.
|
|
|
Unlike ISC DHCP, Kea does track which server in an HA configuration currently holds a free lease, nor is there an efficient way for a server to know if such a lease is in the server's scope, outside of the HA hook library. As this differentiation in state is not required by the RFC, Kea will not be implementing it as this time.
|
|
|
|
|
|
### Lease Store Concurrency
|
|
|
|
|
|
Concurrent access of lease data with MySQL and PostgreSQL is an addressed within
|
|
|
the database servers. As for Memfile, accesses are already mutexed in Memfile_Lease_Mgr
|
|
|
so there should not be issues encountered here either.
|
|
|
Concurrent access of lease data with MySQL and PostgreSQL is an addressed within the database servers. As for Memfile, accesses are already mutexed in Memfile_Lease_Mgr so there should not be issues encountered here either.
|
|
|
|
|
|
There is a quasi-related issue https://gitlab.isc.org/isc-projects/kea/-/issues/2507.
|
|
|
It calls for replacing the use of CriticalSection with mutex locks in the stat-lease<4/6-get
|
|
|
command handlers implemented in the lease-cmds hook library. These commands are used by
|
|
|
customers to periodically fetch statistics. By using a CriticalSection, they needlessly stop
|
|
|
all multi-threaded operations while they process. This would similarly pause all LeaseQuery
|
|
|
processing as well. It would be best to address this issue prior to releasing Bulk LeaseQuery.
|
|
|
There is a quasi-related issue https://gitlab.isc.org/isc-projects/kea/-/issues/2507. It calls for replacing the use of CriticalSection with mutex locks in the stat-lease<4/6-get command handlers implemented in the lease-cmds hook library. These commands are used by customers to periodically fetch statistics. By using a CriticalSection, they needlessly stop all multi-threaded operations while they process. This would similarly pause all LeaseQuery processing as well. It would be best to address this issue prior to releasing Bulk LeaseQuery.
|
|
|
|
|
|
## Task Breakdown
|
|
|
|
|
|
A very preliminary task break down is shown below:
|
|
|
|
|
|
```
|
|
|
1. Query Fulfillment
|
|
|
1.1 Kea core and/or leasequery hook changes to store explicit parameters in user-context
|
... | ... | |