|
|
|
|
|
# High Level Design
|
|
# High Level Design
|
|
|
|
|
|
This section covers the high level design of Bulk LeaseQuery (BLQ) for both DHPCv4
|
|
This section covers the high level design of Bulk LeaseQuery (BLQ) for both DHPCv4 and DHCPv6, as described by [RFC 6926](https://www.rfc-editor.org/rfc/rfc6926) and [RFC 5460](https://www.rfc-editor.org/rfc/rfc5460.html) respectively.
|
|
and DHCPv6, as described by [RFC 6926](https://www.rfc-editor.org/rfc/rfc6926)
|
|
|
|
and [RFC 5460](https://www.rfc-editor.org/rfc/rfc5460.html) respectively.
|
|
|
|
|
|
|
|
## Overview
|
|
## Overview
|
|
|
|
|
|
Bulk LeaseQuery support will be added to the existing LeaseQuery hook
|
|
Bulk LeaseQuery support will be added to the existing LeaseQuery hook and anticipates future extension to support Active LeaseQuery (ALQ). There are some aspects of Basic LeaseQuery implementation that can be reused, though some refactoring of that existing code is likely.
|
|
and anticipates future extension to support Active LeaseQuery (ALQ). There
|
|
|
|
are some aspects of Basic LeaseQuery implementation that can be reused, though
|
|
Running Bulk LeaseQuery will require Kea to be configured in MT mode. BLQ would have significant performance impacts on single-threaded configurations and thus it will not be supported. This should simplify the design as well as the implementation. The LeaseQuery hook library will verify the MT is enabled if BLQ is enabled.
|
|
some refactoring of that existing code is likely.
|
|
|
|
|
|
BLQ will support a configurable number of concurrent TCP connections. Each connection belongs to a single requester who may request multiple queries, each with a XID unique to the connection. A requester need not wait for one query to complete before asking for another. In fact, a single inbound requester packet can contain multiple queries.
|
|
Running Bulk LeaseQuery will require Kea to be configured in MT mode. BLQ
|
|
|
|
would have significant performance impacts on single-threaded configurations
|
|
The RFCs state that servers may process a requester's queries in parallel and intermingle the binding responses sent back, whether the queries arrive in a single request packet or multiple packets. This design provides support for a configurable number of parallel queries per requester.
|
|
and thus it will not be supported. This should simplify the design as well
|
|
|
|
as the implementation. The LeaseQuery hook library will verify the MT is
|
|
There is no inherent difference between a connection for Bulk queries versus a connection for Active queries, other than the requester sending an optional DHCPTLS message and then a DHCPACTIVELEASEQUERY. In theory, a Bulk requester could morph their connection into an Active connection, simply by sending DHCPACTIVELEASEQUERY. While this design does not add anything to directly support ALQ, rather it lays the ground to work add it in the future.
|
|
enabled if BLQ is enabled.
|
|
|
|
|
|
|
|
BLQ will support a configurable number of concurrent TCP connections. Each
|
|
|
|
connection belongs to a single requester who may request multiple queries,
|
|
|
|
each with a XID unique to the connection. A requester need not wait for one
|
|
|
|
query to complete before asking for another. In fact, a single inbound
|
|
|
|
requester packet can contain multiple queries.
|
|
|
|
|
|
|
|
The RFCs state that servers may process a requester's queries in parallel and
|
|
|
|
intermingle the binding responses sent back, whether the queries arrive in a
|
|
|
|
single request packet or multiple packets. This design provides support for a
|
|
|
|
configurable number of parallel queries per requester.
|
|
|
|
|
|
|
|
There is no inherent difference between a connection for Bulk queries versus
|
|
|
|
a connection for Active queries, other than the requester sending an optional
|
|
|
|
DHCPTLS message and then a DHCPACTIVELEASEQUERY. In theory, a Bulk requester
|
|
|
|
could morph their connection into an Active connection, simply by sending
|
|
|
|
DHCPACTIVELEASEQUERY. While this design does not add anything to directly
|
|
|
|
support ALQ, rather it lays the ground to work add it in the future.
|
|
|
|
|
|
|
|
The design is described in the following sections:
|
|
The design is described in the following sections:
|
|
|
|
|
... | @@ -48,29 +26,21 @@ The design is described in the following sections: |
... | @@ -48,29 +26,21 @@ The design is described in the following sections: |
|
|
|
|
|
## Process Flow
|
|
## Process Flow
|
|
|
|
|
|
Unlike normal DHCP or our current REST API traffic, Bulk LeaseQuery is not a typical request/response
|
|
Unlike normal DHCP or our current REST API traffic, Bulk LeaseQuery is not a typical request/response exchange. Each inbound requester packet can contain multiple queries. The outbound server response can contain multiple replies for multiple queries, intermixed, and can span multiple packets.
|
|
exchange. Each inbound requester packet can contain multiple queries. The outbound server response
|
|
|
|
can contain multiple replies for multiple queries, intermixed, and can span multiple packets.
|
|
|
|
|
|
|
|
This design will treat inbound data as a stream of DHCP query requests, each with a unique XID, and where individual TCP packets are simple freight containers conveying them rather than the request half of a transaction. The same perspective will applied to the outbound responses, they will be a stream of bindings and status responses (e.g. LEASEQUERYDONE) each tagged with corresponding XID.
|
|
This design will treat inbound data as a stream of DHCP query requests, each with a unique XID, and where individual TCP packets are simple freight containers conveying them rather than the request half of a transaction. The same perspective will applied to the outbound responses, they will be a stream of bindings and status responses (e.g. LEASEQUERYDONE) each tagged with corresponding XID.
|
|
|
|
|
|
The design introduces a new class, LeaseQueryListener, which will support multiple, concurrent
|
|
The design introduces a new class, LeaseQueryListener, which will support multiple, concurrent TCP connections, one connection per requester, one thread per connection. It will be very similar to and reuse much of the class hierarchy as CmdHttpListener, including TLS support. While a good deal the existing class hierarchy topped by CmdHttpListener can be used, at least as a model for the higher order functions, the HttpConnection::Transaction and the logic using it is not quite the right fit. It is intended to read a single HTTP request that generates a single HTTP response,though in theory, either can span multiple packets. The requisite functions will need to modified accordingly (e.g. doRead(), socketReadCallback()).
|
|
TCP connections, one connection per requester, one thread per connection. It will be very similar to and reuse much of the class hierarchy as CmdHttpListener, including TLS support. While a good deal the existing class hierarchy topped by CmdHttpListener can be used, at least as a model for the higher order functions, the HttpConnection::Transaction and the logic using it is not quite the right fit. It is intended to read a single HTTP request that generates a single HTTP response,though in theory, either can span multiple packets. The requisite functions will need to modified accordingly (e.g. doRead(), socketReadCallback()).
|
|
|
|
|
|
|
|
We can follow the pattern used by LeaseMgr::LeaseStatsQuery to create BulkLeaseQuery class. This class
|
|
We can follow the pattern used by LeaseMgr::LeaseStatsQuery to create BulkLeaseQuery class. This class will will be used for starting a new bulk query and fetching its results. Each lease store will implement its own derivations, and in this way, isolate lease store specifics.
|
|
will will be used for starting a new bulk query and fetching its results. Each lease store will implement
|
|
|
|
its own derivations, and in this way, isolate lease store specifics.
|
|
|
|
|
|
|
|
LeaseQueryListener will provide a connection thread pool in which each thread supports a single
|
|
LeaseQueryListener will provide a connection thread pool in which each thread supports a single connection to one requester. Each connection thread will be responsible for reading incoming query packets and sending outgoing response packets.
|
|
connection to one requester. Each connection thread will be responsible for reading incoming query
|
|
|
|
packets and sending outgoing response packets.
|
|
|
|
|
|
|
|
The following diagram illustrates the new class hierarchy:
|
|
The following diagram illustrates the new class hierarchy:
|
|
|
|
|
|
![blq_classes.svg](uploads/84777a6a72d6a946a6950922d2b08c58/blq_classes.svg)
|
|
![blq_classes.svg](uploads/84777a6a72d6a946a6950922d2b08c58/blq_classes.svg)
|
|
|
|
|
|
When a query packet arrives its queries will be pushed onto a queue. The queue is monitored a pool
|
|
When a query packet arrives its queries will be pushed onto a queue. The queue is monitored a pool by one or more query worker threads. Each worker thread is responsible for submitting a single query against the lease store, iteratively fetching the results, and pushing results as XID/binding pairs to connection's outbound queue. The connection thread will add the binding pairs to the current TCP packet until it is full and then send it asynchronously. Subsequent pairs are adding to a new packet. This continues until there are no more queued queries.
|
|
by one or more query worker threads. Each worker thread is responsible for submitting a single query against the lease store, iteratively fetching the results, and pushing results as XID/binding pairs to connection's outbound queue. The connection thread will add the binding pairs to the current TCP packet until it is full and then send it asynchronously. Subsequent pairs are adding to a new packet. This continues until there are no more queued queries.
|
|
|
|
|
|
|
|
A conceptualization of the processing in pseudo code follows:
|
|
A conceptualization of the processing in pseudo code follows:
|
|
|
|
|
... | @@ -123,66 +93,57 @@ TcpConnection::pushToSend(xid, binding) { |
... | @@ -123,66 +93,57 @@ TcpConnection::pushToSend(xid, binding) { |
|
}
|
|
}
|
|
```
|
|
```
|
|
|
|
|
|
We may need a gating mechanism (e.g. condition variable in pushToSend()), based on
|
|
We may need a gating mechanism (e.g. condition variable in pushToSend()), based on a maximum number of bindings queued for send (per connection) so query workers do not create enormous outbound queues. In other words, we need to avoid outrunning the connection's ability to send by too avoid memory exhausting et al. In practice this may be a non-issue but it is better to allow for it now.
|
|
a maximum number of bindings queued for send (per connection) so query workers do not
|
|
|
|
create enormous outbound queues. In other words, we need to avoid outrunning the
|
|
|
|
connection's ability to send by too avoid memory exhausting et al. In practice this
|
|
|
|
may be a non-issue but it is better to allow for it now.
|
|
|
|
|
|
|
|
## Query Execution
|
|
## Query Execution
|
|
|
|
|
|
It is possible for queries to select very large numbers of leases, certainly
|
|
It is possible for queries to select very large numbers of leases, certainly far more than could be returned in a single fetch. While this is not a necessarily a problem for Memfile lease storage, it represents a significant consideration for DB lease storage. This can be approached at least two ways: use cursors or use iterative queries.
|
|
far more than could be returned in a single fetch. While this is not a necessarily
|
|
|
|
a problem for Memfile lease storage, it represents a significant consideration
|
|
|
|
for DB lease storage. This can be approached at least two ways: use cursors or use
|
|
|
|
iterative queries.
|
|
|
|
|
|
|
|
### Cursors
|
|
### Cursors
|
|
|
|
|
|
With cursors the desired query is submitted to the DB once, and the results
|
|
With cursors the desired query is submitted to the DB once, and the results are fetched piece meal. While we have not used this approach within Kea yet, it maybe it does place the burden or performance on the DB server.
|
|
are fetched piece meal. While we have not used this approach within Kea yet,
|
|
|
|
it maybe it does place the burden or performance on the DB server.
|
|
|
|
|
|
|
|
#### Postgresql
|
|
**Postgresql**
|
|
|
|
|
|
Cursors are identified by name in Posgresql, names need only be unique
|
|
```
|
|
within a session. The cursor must first be created for the given SQL
|
|
Cursors are identified by name in Posgresql, names need only be unique
|
|
statement, and then used in iterative fetch invocations to retrieve
|
|
within a session. The cursor must first be created for the given SQL
|
|
the query results by fetching the NEXT n records:
|
|
statement, and then used in iterative fetch invocations to retrieve
|
|
|
|
the query results by fetching the NEXT n records:
|
|
|
|
```
|
|
|
|
|
|
```
|
|
```
|
|
FETCH NEXT n FROM cursor INTO rowvar
|
|
FETCH NEXT n FROM cursor INTO rowvar
|
|
```
|
|
```
|
|
as illustrated here:
|
|
|
|
|
|
|
|
https://www.postgresql.org/docs/current/plpgsql-cursors.html
|
|
```
|
|
|
|
as illustrated here:
|
|
|
|
|
|
(SIDEBAR - cursors might come in handy for use with expiration some day)
|
|
https://www.postgresql.org/docs/current/plpgsql-cursors.html
|
|
|
|
|
|
#### MySQL
|
|
(SIDEBAR - cursors might come in handy for use with expiration some day)
|
|
|
|
```
|
|
|
|
|
|
|
|
**MySQL**
|
|
|
|
|
|
To use cursors with a prepared statement we need to set requisite statement attributes
|
|
```
|
|
with mysq_stmt_attr_set(). Note, that one of those parameters, STMT_ATTR_PREFETCH_ROWS,
|
|
To use cursors with a prepared statement we need to set requisite statement attributes
|
|
determines the number of rows returned on each fetch:
|
|
with mysq_stmt_attr_set(). Note, that one of those parameters, STMT_ATTR_PREFETCH_ROWS,
|
|
|
|
determines the number of rows returned on each fetch:
|
|
|
|
|
|
https://dev.mysql.com/doc/c-api/5.6/en/mysql-stmt-attr-set.html
|
|
https://dev.mysql.com/doc/c-api/5.6/en/mysql-stmt-attr-set.html
|
|
|
|
|
|
Statement ID serves as cursor identifier. Do not call mysql_stmt_store_result(),
|
|
Statement ID serves as cursor identifier. Do not call mysql_stmt_store_result(),
|
|
but make repeated calls to mysql_stmt_fetch(). I THINK that behind the scenes
|
|
but make repeated calls to mysql_stmt_fetch(). I THINK that behind the scenes
|
|
mysql client lib returns to server when you exhaust the local fetched records,
|
|
mysql client lib returns to server when you exhaust the local fetched records,
|
|
rather then for every row.
|
|
rather then for every row.
|
|
|
|
```
|
|
|
|
|
|
### Iterative queries
|
|
### Iterative queries
|
|
|
|
|
|
The queries would order by ip-address and apply either a starting address or
|
|
The queries would order by ip-address and apply either a starting address or a numerical offset and a row limit. When the current fetched results has been consumed, a subsequent query is issued asking for rows with addresses > then the maximum address of the previous fetch. The initial query would use a beginning address of 0.
|
|
a numerical offset and a row limit. When the current fetched results has been consumed,
|
|
|
|
a subsequent query is issued asking for rows with addresses > then the maximum address of the
|
|
|
|
previous fetch. The initial query would use a beginning address of 0.
|
|
|
|
|
|
|
|
This is akin to doing our own cursors but differs in performance portent, they may be
|
|
This is akin to doing our own cursors but differs in performance portent, they may be less efficient. Each call is a new SQL execution, in addition to needing to sort the results by IP address (though we have existing index for this). Example selects for either method for PostgreSQL are shown below (MySQL would be similar):
|
|
less efficient. Each call is a new SQL execution, in addition to needing to sort the
|
|
|
|
results by IP address (though we have existing index for this). Example selects for
|
|
|
|
either method for PostgreSQL are shown below (MySQL would be similar):
|
|
|
|
|
|
|
|
Using a starting IP address ("$1" is always the last IP address returned from the prior query):
|
|
Using a starting IP address ("$1" is always the last IP address returned from the prior query):
|
|
|
|
|
... | @@ -197,9 +158,7 @@ ORDER by ip-addr |
... | @@ -197,9 +158,7 @@ ORDER by ip-addr |
|
LIMIT $3
|
|
LIMIT $3
|
|
```
|
|
```
|
|
|
|
|
|
Or per this page https://www.postgresql.org/docs/12/queries-limit.html, you can
|
|
Or per this page https://www.postgresql.org/docs/12/queries-limit.html, you can use skip the first n rows by using OFFSET and LIMIT together with ORDER BY:
|
|
use skip the first n rows by using OFFSET and LIMIT together with ORDER BY:
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
```
|
|
SELECT *
|
|
SELECT *
|
... | @@ -210,25 +169,19 @@ ORDER by ip-addr |
... | @@ -210,25 +169,19 @@ ORDER by ip-addr |
|
LIMIT $2
|
|
LIMIT $2
|
|
OFFSET $3
|
|
OFFSET $3
|
|
```
|
|
```
|
|
|
|
|
|
where "$3" is the total number of rows already returned.
|
|
where "$3" is the total number of rows already returned.
|
|
|
|
|
|
This does imply that subsequent query invocations would repeat some portion of the
|
|
This does imply that subsequent query invocations would repeat some portion of the record selection of previous queries. Consider moving through 100K leases 10K at a time. You have to skip 10K, then 20K, then 30K records and then start fetching against the LIMIT. Most DB servers do retain some amount of caching per session, so this may be something of a non-issue
|
|
record selection of previous queries. Consider moving through 100K leases 10K at a time.
|
|
|
|
You have to skip 10K, then 20K, then 30K records and then start fetching against
|
|
|
|
the LIMIT. Most DB servers do retain some amount of caching per session, so this may be
|
|
|
|
something of a non-issue
|
|
|
|
|
|
|
|
On the positive side, using this technique is something we already do with the
|
|
On the positive side, using this technique is something we already do with the leaseX-get-page commands in lease_cmds hook library. Additionally, the SELECT and results fetch are effectively one operation, perhaps reducing the chance for concurrent lease changes.
|
|
leaseX-get-page commands in lease_cmds hook library. Additionally, the SELECT and
|
|
|
|
results fetch are effectively one operation, perhaps reducing the chance for
|
|
|
|
concurrent lease changes.
|
|
|
|
|
|
|
|
## Query Data Requirements
|
|
## Query Data Requirements
|
|
|
|
|
|
Looking more specifically at the query data requirements, a list of the queries called
|
|
Looking more specifically at the query data requirements, a list of the queries called for by the BLQ RFCs follows:
|
|
for by the BLQ RFCs follows:
|
|
|
|
|
|
**V4 Queries**
|
|
|
|
|
|
V4 Queries:
|
|
|
|
1. By MAC
|
|
1. By MAC
|
|
2. By Client ID
|
|
2. By Client ID
|
|
3. By Relay ID (client supplies via RAI sub option 12)
|
|
3. By Relay ID (client supplies via RAI sub option 12)
|
... | @@ -236,18 +189,15 @@ V4 Queries: |
... | @@ -236,18 +189,15 @@ V4 Queries: |
|
5. By VPN ID (client supplies via VPN select option, see RFC 6607) - if we choose to support it
|
|
5. By VPN ID (client supplies via VPN select option, see RFC 6607) - if we choose to support it
|
|
6. All Configured IP Addresses
|
|
6. All Configured IP Addresses
|
|
|
|
|
|
V6 Queries:
|
|
**V6 Queries**
|
|
1. Relay ID (option 53 D60_RELAY_ID extracted from Pkt6::relay_info_.options_)
|
|
|
|
|
|
1. Relay ID (option 53 D60_RELAY_ID extracted from Pkt6::relay_info\_.options\_)
|
|
2. Link address - from existing extended relay-info
|
|
2. Link address - from existing extended relay-info
|
|
3. Remote ID (option 37 D60_REMOTE_ID extracted from RelayInfo)
|
|
3. Remote ID (option 37 D60_REMOTE_ID extracted from RelayInfo)
|
|
|
|
|
|
In order to query for leases by these parameters, their values need to be readily accessible
|
|
In order to query for leases by these parameters, their values need to be readily accessible for each lease within the lease store.
|
|
for each lease within the lease store.
|
|
|
|
|
|
|
|
When Basic LeaseQuery was implemented, it added a global configuration
|
|
When Basic LeaseQuery was implemented, it added a global configuration parameter, "store-extended-info", which enables Kea core code to store relay information for each lease. For V4 this consists of the RAI option (82) stored as a hex literal:
|
|
parameter, "store-extended-info", which enables Kea core code to store
|
|
|
|
relay information for each lease. For V4 this consists of the RAI option (82)
|
|
|
|
stored as a hex literal:
|
|
|
|
|
|
|
|
```
|
|
```
|
|
"ISC":
|
|
"ISC":
|
... | @@ -274,9 +224,7 @@ and for V6, we store an entry for each relay layer in an array: |
... | @@ -274,9 +224,7 @@ and for V6, we store an entry for each relay layer in an array: |
|
}
|
|
}
|
|
```
|
|
```
|
|
|
|
|
|
Querying for leases based on option values that are embedded in the packed data
|
|
Querying for leases based on option values that are embedded in the packed data stored as "options" is untenable. Therefore, the design proposes adding individual values for each parameter to the user-context.
|
|
stored as "options" is untenable. Therefore, the design proposes adding individual
|
|
|
|
values for each parameter to the user-context.
|
|
|
|
|
|
|
|
For V4, the context would appear as follows:
|
|
For V4, the context would appear as follows:
|
|
|
|
|
... | @@ -288,7 +236,6 @@ For V4, the context would appear as follows: |
... | @@ -288,7 +236,6 @@ For V4, the context would appear as follows: |
|
"relay-id" : 0x... // extracted from RAI
|
|
"relay-id" : 0x... // extracted from RAI
|
|
...
|
|
...
|
|
}
|
|
}
|
|
|
|
|
|
```
|
|
```
|
|
|
|
|
|
and for v6:
|
|
and for v6:
|
... | @@ -311,21 +258,12 @@ and for v6: |
... | @@ -311,21 +258,12 @@ and for v6: |
|
}
|
|
}
|
|
```
|
|
```
|
|
|
|
|
|
These new values could be populated in the lease's user_context in at least two
|
|
These new values could be populated in the lease's user_context in at least two ways:
|
|
ways:
|
|
|
|
|
|
|
|
1. Done in Kea core in AllocEngine::updateLease<4/6>ExtendedInfo() functions.
|
|
1. Done in Kea core in AllocEngine::updateLease<4/6>ExtendedInfo() functions. These two functions only store lease information in the user-context when the "store-extended-info" parameter is true. We would simply be adding a bit more code here.
|
|
These two functions only store lease information in the user-context when
|
|
2. Within call outs for hook points such as lease<4/6>\_select, lease<4/6>\_renew that BLQ would implement. These call outs could also subsume the code now done n updateLease<4/6>ExtendedInfo() in addition to adding the new explicit values.
|
|
the "store-extended-info" parameter is true. We would simply be adding a bit
|
|
|
|
more code here.
|
|
|
|
|
|
|
|
2. Within call outs for hook points such as lease<4/6>_select, lease<4/6>_renew
|
|
At this point, regardless of the lease store used, the LeaseX::user_context\_ would contain individual JSON key/value pairs for all of the BLQ query parameters. However, this alone is not enough to allow for efficient querying.
|
|
that BLQ would implement. These call outs could also subsume the code now done n
|
|
|
|
updateLease<4/6>ExtendedInfo() in addition to adding the new explicit values.
|
|
|
|
|
|
|
|
At this point, regardless of the lease store used, the LeaseX::user_context_ would
|
|
|
|
contain individual JSON key/value pairs for all of the BLQ query parameters. However,
|
|
|
|
this alone is not enough to allow for efficient querying.
|
|
|
|
|
|
|
|
### V4 Query time parameters, STOS, and CLTT
|
|
### V4 Query time parameters, STOS, and CLTT
|
|
|
|
|
... | @@ -353,15 +291,9 @@ and also Sec 8.3, item 3: |
... | @@ -353,15 +291,9 @@ and also Sec 8.3, item 3: |
|
dhcp-state option's state became valid.
|
|
dhcp-state option's state became valid.
|
|
```
|
|
```
|
|
|
|
|
|
Kea does not store state-time-of-state (STOS) as does ISC DHCP and CLTT is derived by
|
|
Kea does not store state-time-of-state (STOS) as does ISC DHCP and CLTT is derived by calculation: expiration - valid_lft. In other words, currently Kea has a derived hybrid of STOS/CLTT. First, we would probably need to document that Kea does not currently track STOS, and as such only base query results on our notion of CLTT.
|
|
calculation: expiration - valid_lft. In other words, currently Kea has a derived hybrid of
|
|
|
|
STOS/CLTT. First, we would probably need to document that Kea does not currently track
|
|
|
|
STOS, and as such only base query results on our notion of CLTT.
|
|
|
|
|
|
|
|
Second, in order to avoid full table scans, we would likely need an index on a
|
|
Second, in order to avoid full table scans, we would likely need an index on a generated column for the calculation. We will need a function to generate such a column, as we must account for infinite lifetime leases, as there is a bit of trickery done when such a lease is stored:
|
|
generated column for the calculation. We will need a function to generate such a
|
|
|
|
column, as we must account for infinite lifetime leases, as there is a bit of
|
|
|
|
trickery done when such a lease is stored:
|
|
|
|
|
|
|
|
```
|
|
```
|
|
// The lease structure holds the client last transmission time (cltt_)
|
|
// The lease structure holds the client last transmission time (cltt_)
|
... | @@ -378,7 +310,7 @@ trickery done when such a lease is stored: |
... | @@ -378,7 +310,7 @@ trickery done when such a lease is stored: |
|
}
|
|
}
|
|
```
|
|
```
|
|
|
|
|
|
Thus the stored value for expire_ is already CLTT.
|
|
Thus the stored value for expire\_ is already CLTT.
|
|
|
|
|
|
### V4 Query for All Configured IP Addresses
|
|
### V4 Query for All Configured IP Addresses
|
|
|
|
|
... | @@ -390,6 +322,7 @@ RFC6929/Sec 7.2: |
... | @@ -390,6 +322,7 @@ RFC6929/Sec 7.2: |
|
A Query for All Configured IP addresses is signaled by the absence
|
|
A Query for All Configured IP addresses is signaled by the absence
|
|
of any other primary query.
|
|
of any other primary query.
|
|
```
|
|
```
|
|
|
|
|
|
and section 8.2:
|
|
and section 8.2:
|
|
|
|
|
|
```
|
|
```
|
... | @@ -399,9 +332,7 @@ and section 8.2: |
... | @@ -399,9 +332,7 @@ and section 8.2: |
|
binding to a DHCPv4 client MUST be returned in a DHCPLEASEUNASSIGNED message.
|
|
binding to a DHCPv4 client MUST be returned in a DHCPLEASEUNASSIGNED message.
|
|
```
|
|
```
|
|
|
|
|
|
Thus kea-dhcp4 will need to extrapolate all of the free leases based on pool
|
|
Thus kea-dhcp4 will need to extrapolate all of the free leases based on pool knowledge. Remembering that assigned leases are fetched, ordered by lease address, then the algorithm to process the results might look something like this:
|
|
knowledge. Remembering that assigned leases are fetched, ordered by lease address,
|
|
|
|
then the algorithm to process the results might look something like this:
|
|
|
|
|
|
|
|
```
|
|
```
|
|
current_pool = none
|
|
current_pool = none
|
... | @@ -458,54 +389,35 @@ if (last_sent < pool_end) { |
... | @@ -458,54 +389,35 @@ if (last_sent < pool_end) { |
|
pushToSend(XID, free_lease);
|
|
pushToSend(XID, free_lease);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
|
|
```
|
|
```
|
|
|
|
|
|
There is also some questions as to how to treat host reservations for fixed addresses.
|
|
There is also some questions as to how to treat host reservations for fixed addresses. Those that are in-pool but without a lease would be sent back as free by extrapolation. Technically accurate in terms of lease state but functionally it is only available to a single client. For out-of-pool addresses without leases, we would send back nothing without doing some sort of explicit search for them.
|
|
Those that are in-pool but without a lease would be sent back as free by extrapolation.
|
|
|
|
Technically accurate in terms of lease state but functionally it is only available to
|
|
|
|
a single client. For out-of-pool addresses without leases, we would send back nothing
|
|
|
|
without doing some sort of explicit search for them.
|
|
|
|
|
|
|
|
In order to avoid potentially costly searches against host reservations, it is proposed
|
|
In order to avoid potentially costly searches against host reservations, it is proposed that we simply document the above as the expected behavior.
|
|
that we simply document the above as the expected behavior.
|
|
|
|
|
|
|
|
### V4 Query by VPN ID
|
|
### V4 Query by VPN ID
|
|
|
|
|
|
As has been suggested by @fdupont: "VPNs are already not supported by Kea, e.g. you can't define multiple subnets using 10.0.0.0/8 private range saying they are in different VPNs (note we can do this one day: I simply believe we wait for an explicit request from a customer...)".
|
|
As has been suggested by @fdupont: "VPNs are already not supported by Kea, e.g. you can't define multiple subnets using 10.0.0.0/8 private range saying they are in different VPNs (note we can do this one day: I simply believe we wait for an explicit request from a customer...)".\
|
|
There is little value in trying to support it in BLQ at this juncture.
|
|
There is little value in trying to support it in BLQ at this juncture.
|
|
|
|
|
|
## Query Mechanics
|
|
## Query Mechanics
|
|
|
|
|
|
Initially, it looked as though PostgreSQL and MySQL both supported sufficient JSON
|
|
Initially, it looked as though PostgreSQL and MySQL both supported sufficient JSON mechanisms to index and query by JSON. However, further exploration revealed the following:
|
|
mechanisms to index and query by JSON. However, further exploration revealed the
|
|
|
|
following:
|
|
|
|
|
|
|
|
1. PostgreSQL does support Generalized Inverted (GIN) indexes on JSONB columns, and
|
|
|
|
those mechanisms appear sufficient to address both V4 and V6 needs. However,
|
|
|
|
JSONB columns are not supported until PostgreSQL 9.4. In order to use GIN indexes
|
|
|
|
the user_context column type in the leaseX tables would have to be changed from TEXT
|
|
|
|
to JSONB. This would necessitate changing many of the existing SQL statements in
|
|
|
|
postgresql lease manager. Furthermore, it would mean Kea's PostgreSQL
|
|
|
|
schema could no longer with PostgreSQL pre 9.4.
|
|
|
|
|
|
|
|
2. MySQL does not support indexing on JSON columns at all, in any version. Nor do
|
|
1. PostgreSQL does support Generalized Inverted (GIN) indexes on JSONB columns, and those mechanisms appear sufficient to address both V4 and V6 needs. However, JSONB columns are not supported until PostgreSQL 9.4. In order to use GIN indexes the user_context column type in the leaseX tables would have to be changed from TEXT to JSONB. This would necessitate changing many of the existing SQL statements in postgresql lease manager. Furthermore, it would mean Kea's PostgreSQL schema could no longer with PostgreSQL pre 9.4.
|
|
they seem likely to do so. For indexed queries (i.e not full table scans) would
|
|
2. MySQL does not support indexing on JSON columns at all, in any version. Nor do they seem likely to do so. For indexed queries (i.e not full table scans) would require using generated columns, explicit columns, or a cross-reference table(s).
|
|
require using generated columns, explicit columns, or a cross-reference table(s).
|
|
|
|
|
|
|
|
### V4 solutions
|
|
### V4 solutions
|
|
|
|
|
|
#### Memfile
|
|
**Memfile**
|
|
Lease::user_context_ is an ElementPtrUsing so it should be possible to create predicate
|
|
|
|
objects to extract the query value to create new indexes for the lease multi-index-container
|
|
Lease::user_context\_ is an ElementPtrUsing so it should be possible to create predicate objects to extract the query value to create new indexes for the lease multi-index-container such as is described here:
|
|
such as is described here:
|
|
|
|
|
|
|
|
https://stackoverflow.com/questions/1628321/boost-multi-index-container-with-index-based-on-nested-values
|
|
https://stackoverflow.com/questions/1628321/boost-multi-index-container-with-index-based-on-nested-values
|
|
|
|
|
|
#### MySQL
|
|
**MySQL**
|
|
|
|
|
|
Use a generated column for each query value. This would require either a function or
|
|
Use a generated column for each query value. This would require either a function or an expression to extract the value from user_context, similar to this:
|
|
an expression to extract the value from user_context, similar to this:
|
|
|
|
|
|
|
|
```
|
|
```
|
|
gen_relay_id varbinary(128) GENERATED ALWAYS AS (getRelayIdFromUserContext(user_context)),
|
|
gen_relay_id varbinary(128) GENERATED ALWAYS AS (getRelayIdFromUserContext(user_context)),
|
... | @@ -515,14 +427,11 @@ an expression to extract the value from user_context, similar to this: |
... | @@ -515,14 +427,11 @@ an expression to extract the value from user_context, similar to this: |
|
...
|
|
...
|
|
```
|
|
```
|
|
|
|
|
|
Generated columns are automatically calculated on record INSERT and UPDATE, without the extra
|
|
Generated columns are automatically calculated on record INSERT and UPDATE, without the extra code involved with adding new explicit columns that are populated by triggers.
|
|
code involved with adding new explicit columns that are populated by triggers.
|
|
|
|
|
|
|
|
#### PostgreSQL:
|
|
**PostgreSQL**
|
|
|
|
|
|
Assuming we choose to use JSONB/GIN some examples are select statements are worth exploring.
|
|
Assuming we choose to use JSONB/GIN some examples are select statements are worth exploring. These statements include data that is similar in structure to V6 user-context content and uses a JSONB boolean expression similar to what we would employ for BLQ. The followings statement returns true (t):
|
|
These statements include data that is similar in structure to V6 user-context content and uses
|
|
|
|
a JSONB boolean expression similar to what we would employ for BLQ. The followings statement returns true (t):
|
|
|
|
|
|
|
|
```
|
|
```
|
|
select '{ "relay-info":[ { "remote-id":10, "hop":1 }, { "remote-id":20, "hop":2 }]}'::jsonb @> '{"relay-info":[{"remote-id":20}]}'::jsonb;
|
|
select '{ "relay-info":[ { "remote-id":10, "hop":1 }, { "remote-id":20, "hop":2 }]}'::jsonb @> '{"relay-info":[{"remote-id":20}]}'::jsonb;
|
... | @@ -536,8 +445,7 @@ select '{ "relay-info":[ { "remote-id":10, "hop":1 }, { "remote-id":20, "hop":2 |
... | @@ -536,8 +445,7 @@ select '{ "relay-info":[ { "remote-id":10, "hop":1 }, { "remote-id":20, "hop":2 |
|
select '{ "no-relay-info": "nada" }'::jsonb @> '{"relay-info":[{"remote-id":20}]}'::jsonb;
|
|
select '{ "no-relay-info": "nada" }'::jsonb @> '{"relay-info":[{"remote-id":20}]}'::jsonb;
|
|
```
|
|
```
|
|
|
|
|
|
The user_context column for both lease4 and lease6 would need to be changed to JSONB and we will need
|
|
The user_context column for both lease4 and lease6 would need to be changed to JSONB and we will need to add GIN (Generalized Inverted Index) indexes for them:
|
|
to add GIN (Generalized Inverted Index) indexes for them:
|
|
|
|
|
|
|
|
```
|
|
```
|
|
ALTER TABLE lease4 ALTER COLUMN user_context TYPE JSONB using user_context::jsonb;
|
|
ALTER TABLE lease4 ALTER COLUMN user_context TYPE JSONB using user_context::jsonb;
|
... | @@ -546,22 +454,17 @@ ALTER TABLE lease6 ALTER COLUMN user_context TYPE JSONB using user_context::json |
... | @@ -546,22 +454,17 @@ ALTER TABLE lease6 ALTER COLUMN user_context TYPE JSONB using user_context::json |
|
CREATE INDEX lease6_user_context_gin ON lease6 USING GIN (user_context);
|
|
CREATE INDEX lease6_user_context_gin ON lease6 USING GIN (user_context);
|
|
```
|
|
```
|
|
|
|
|
|
Note it will be necessary to do more robust performance testing to formulate the queries and the GIN
|
|
Note it will be necessary to do more robust performance testing to formulate the queries and the GIN indexes. The above code is demonstrative. PostgreSQL GIN index expressions are fairly rich and we should be able to tailor them to meet differing needs between V4 and V6 to maximize performance.
|
|
indexes. The above code is demonstrative. PostgreSQL GIN index expressions are fairly rich and
|
|
|
|
we should be able to tailor them to meet differing needs between V4 and V6 to maximize performance.
|
|
|
|
|
|
|
|
If we choose not to use JSONB/GIN, we can use the generated column approach here too.
|
|
If we choose not to use JSONB/GIN, we can use the generated column approach here too. PostgreSQL supports it and it would avoid incompatibility with older PostgreSQL versions.
|
|
PostgreSQL supports it and it would avoid incompatibility with older PostgreSQL versions.
|
|
|
|
|
|
|
|
### V6 solutions
|
|
### V6 solutions
|
|
|
|
|
|
V6 adds a many-to-many wrinkle to the mix: Each lease could pass through multiple relay layers,
|
|
V6 adds a many-to-many wrinkle to the mix: Each lease could pass through multiple relay layers, each of which has its own link address and may add relay-id, remote-id or both (among other options).
|
|
each of which has its own link address and may add relay-id, remote-id or both (among other options).
|
|
|
|
|
|
|
|
#### MySQL
|
|
**MySQL**
|
|
|
|
|
|
For MySQL the only reasonable way I can see is to create a cross-reference table that is maintained
|
|
For MySQL the only reasonable way I can see is to create a cross-reference table that is maintained by triggers on create, update, and delete lease. The table might look something like this:
|
|
by triggers on create, update, and delete lease. The table might look something like this:
|
|
|
|
|
|
|
|
```
|
|
```
|
|
CREATE TABLE IF NOT EXISTS lease6_relay {
|
|
CREATE TABLE IF NOT EXISTS lease6_relay {
|
... | @@ -580,35 +483,19 @@ CREATE TABLE IF NOT EXISTS lease6_relay { |
... | @@ -580,35 +483,19 @@ CREATE TABLE IF NOT EXISTS lease6_relay { |
|
};
|
|
};
|
|
```
|
|
```
|
|
|
|
|
|
#### PostgreSQL
|
|
**PostgreSQL**
|
|
|
|
|
|
If we choose not to employ JSONB/GIN, the we would need to create a similar cross-reference
|
|
If we choose not to employ JSONB/GIN, the we would need to create a similar cross-reference table as done for MySQL.
|
|
table as done for MySQL.
|
|
|
|
|
|
|
|
For either DB, it would be relatively difficult to populate the cross-reference table from
|
|
For either DB, it would be relatively difficult to populate the cross-reference table from existing lease data during an upgrade. The existing leases' user-contexts would have to include extended information stored from which to extract the values. Parsing those value in shell or SQL would be rather ugly (though possible). It would likely be best to treat this as we lease limits, and declare that it will only work for leases assigned or renewed after BLQ has been installed.
|
|
existing lease data during an upgrade. The existing leases' user-contexts would have to include
|
|
|
|
extended information stored from which to extract the values. Parsing those value in shell
|
|
|
|
or SQL would be rather ugly (though possible). It would likely be best to treat this as we
|
|
|
|
lease limits, and declare that it will only work for leases assigned or renewed after BLQ has been
|
|
|
|
installed.
|
|
|
|
|
|
|
|
#### Memfile
|
|
**Memfile**
|
|
|
|
|
|
For Memfile_LeaseMgr, we would mimic the cross-reference table with a new multi_index_container
|
|
For Memfile_LeaseMgr, we would mimic the cross-reference table with a new multi_index_container and maintain it by adding calls to maintenance functions inside addLease(), updateLease(), and deleteLease(), similar to what was done to support lease limits. The function calls could be skipped if store-extended-info is false. The cross-reference container would need to be initialized during dhcp_srv_configured callback, as loading Memfile stores leases directly into the container, rather than calling addLease(). This would only occur when BLQ is enabled. Note the container contents are not persisted across reboots.
|
|
and maintain it by adding calls to maintenance functions inside addLease(), updateLease(), and
|
|
|
|
deleteLease(), similar to what was done to support lease limits. The function calls could be
|
|
|
|
skipped if store-extended-info is false. The cross-reference container would need to be initialized
|
|
|
|
during dhcp_srv_configured callback, as loading Memfile stores leases directly into the container,
|
|
|
|
rather than calling addLease(). This would only occur when BLQ is enabled. Note the container
|
|
|
|
contents are not persisted across reboots.
|
|
|
|
|
|
|
|
## Configuration
|
|
## Configuration
|
|
|
|
|
|
Current Lease Query hook library config has a single parameter, ``requesters`` which is
|
|
Current Lease Query hook library config has a single parameter, `requesters` which is a list of addresses from whom Basic LeaseQuery messages are accepted. Bulk LeaseQuery will require some additions. Because Active LeaseQuery builds upon Bulk LeaseQuery, the additions have been lumped into a new element called "advanced" and is depicted below:
|
|
a list of addresses from whom Basic LeaseQuery messages are accepted. Bulk LeaseQuery
|
|
|
|
will require some additions. Because Active LeaseQuery builds upon Bulk LeaseQuery,
|
|
|
|
the additions have been lumped into a new element called "advanced" and is depicted
|
|
|
|
below:
|
|
|
|
|
|
|
|
```
|
|
```
|
|
"hooks-libraries": [
|
|
"hooks-libraries": [
|
... | @@ -639,27 +526,15 @@ below: |
... | @@ -639,27 +526,15 @@ below: |
|
],
|
|
],
|
|
```
|
|
```
|
|
|
|
|
|
* "bulk-query-enabled" - when true, Kea will accept connections from IPs in the requesters list
|
|
* "bulk-query-enabled" - when true, Kea will accept connections from IPs in the requesters list and honor process BLQ queries received
|
|
and honor process BLQ queries received
|
|
* "active-query-enabled" - when true, Kea will accept connections from IPs in the requesters list and allow establishment of active query sessions. Anticipated parameter, ALQ is not YET covered by this design.
|
|
|
|
|
|
* "active-query-enabled" - when true, Kea will accept connections from IPs in the requesters
|
|
|
|
list and allow establishment of active query sessions. Anticipated parameter,
|
|
|
|
ALQ is not YET covered by this design.
|
|
|
|
|
|
|
|
* "lease-query-ip" - ip address upon which to listen for advanced query connections
|
|
* "lease-query-ip" - ip address upon which to listen for advanced query connections
|
|
* "lease-query-port - port upon which to listen, defaults to 67 for v4 and 547 for v6
|
|
* "lease-query-port - port upon which to listen, defaults to 67 for v4 and 547 for v6
|
|
|
|
|
|
* "max-requester-connections" - maximum number of concurrent requester connections (must be > 0)
|
|
* "max-requester-connections" - maximum number of concurrent requester connections (must be > 0)
|
|
|
|
* "max-concurrent-queries" - maximum number of concurrent queries per connection. A value 0 leaves the number for Kea to determine.
|
|
* "max-concurrent-queries" - maximum number of concurrent queries per connection. A value 0 leaves
|
|
* "max-requester-idle-time" - amount time that may elapse between receiving data from a requester before its connection is closed as idle.
|
|
the number for Kea to determine.
|
|
|
|
|
|
|
|
* "max-requester-idle-time" - amount time that may elapse between receiving data from a requester
|
|
|
|
before its connection is closed as idle.
|
|
|
|
|
|
|
|
* "max-send-queue-size" - maximum number of outbound bindings queued to send (per connection)
|
|
* "max-send-queue-size" - maximum number of outbound bindings queued to send (per connection)
|
|
|
|
* "max-leases-per-fetch" - maximum number of leases to return in a single fetch
|
|
* "max-leases-per-fetch" - maximum number of leases to return in a single fetch
|
|
|
|
|
|
|
|
The TLS parameters are self-explanatory.
|
|
The TLS parameters are self-explanatory.
|
|
|
|
|
... | @@ -667,31 +542,20 @@ The TLS parameters are self-explanatory. |
... | @@ -667,31 +542,20 @@ The TLS parameters are self-explanatory. |
|
|
|
|
|
### dhcp-state "available" vs "remote"
|
|
### dhcp-state "available" vs "remote"
|
|
|
|
|
|
RFC 6926/Sec 6.2.7, talks about lease states of "available" or "remote" when they
|
|
RFC 6926/Sec 6.2.7, talks about lease states of "available" or "remote" when they are unassigned based upon whether or not they are currently in scope for (i.e. held by) the server responding to the LeaseQuery.
|
|
are unassigned based upon whether or not they are currently in scope for (i.e. held
|
|
|
|
by) the server responding to the LeaseQuery.
|
|
|
|
|
|
|
|
Unlike ISC DHCP, Kea does track which server in an HA configuration currently holds
|
|
Unlike ISC DHCP, Kea does track which server in an HA configuration currently holds a free lease, nor is there an efficient way for a server to know if such a lease is in the server's scope, outside of the HA hook library. As this differentiation in state is not required by the RFC, Kea will not be implementing it as this time.
|
|
a free lease, nor is there an efficient way for a server to know if such a lease is
|
|
|
|
in the server's scope, outside of the HA hook library. As this differentiation
|
|
|
|
in state is not required by the RFC, Kea will not be implementing it as this time.
|
|
|
|
|
|
|
|
### Lease Store Concurrency
|
|
### Lease Store Concurrency
|
|
|
|
|
|
Concurrent access of lease data with MySQL and PostgreSQL is an addressed within
|
|
Concurrent access of lease data with MySQL and PostgreSQL is an addressed within the database servers. As for Memfile, accesses are already mutexed in Memfile_Lease_Mgr so there should not be issues encountered here either.
|
|
the database servers. As for Memfile, accesses are already mutexed in Memfile_Lease_Mgr
|
|
|
|
so there should not be issues encountered here either.
|
|
|
|
|
|
|
|
There is a quasi-related issue https://gitlab.isc.org/isc-projects/kea/-/issues/2507.
|
|
There is a quasi-related issue https://gitlab.isc.org/isc-projects/kea/-/issues/2507. It calls for replacing the use of CriticalSection with mutex locks in the stat-lease<4/6-get command handlers implemented in the lease-cmds hook library. These commands are used by customers to periodically fetch statistics. By using a CriticalSection, they needlessly stop all multi-threaded operations while they process. This would similarly pause all LeaseQuery processing as well. It would be best to address this issue prior to releasing Bulk LeaseQuery.
|
|
It calls for replacing the use of CriticalSection with mutex locks in the stat-lease<4/6-get
|
|
|
|
command handlers implemented in the lease-cmds hook library. These commands are used by
|
|
|
|
customers to periodically fetch statistics. By using a CriticalSection, they needlessly stop
|
|
|
|
all multi-threaded operations while they process. This would similarly pause all LeaseQuery
|
|
|
|
processing as well. It would be best to address this issue prior to releasing Bulk LeaseQuery.
|
|
|
|
|
|
|
|
## Task Breakdown
|
|
## Task Breakdown
|
|
|
|
|
|
A very preliminary task break down is shown below:
|
|
A very preliminary task break down is shown below:
|
|
|
|
|
|
```
|
|
```
|
|
1. Query Fulfillment
|
|
1. Query Fulfillment
|
|
1.1 Kea core and/or leasequery hook changes to store explicit parameters in user-context
|
|
1.1 Kea core and/or leasequery hook changes to store explicit parameters in user-context
|
... | | ... | |