Kea issueshttps://gitlab.isc.org/isc-projects/kea/-/issues2023-07-17T13:58:24Zhttps://gitlab.isc.org/isc-projects/kea/-/issues/1654Leases-write command2023-07-17T13:58:24ZTomek MrugalskiLeases-write command There was an issue one customer experienced. He was using memfile and ran out of diskspace. Kea kept running, but the leases were only updated in memory and not written to disk.
Currently, there is no way to recover from this situation... There was an issue one customer experienced. He was using memfile and ran out of diskspace. Kea kept running, but the leases were only updated in memory and not written to disk.
Currently, there is no way to recover from this situation. The ``leases-write`` command would allow manually forcing writing all leases to disk.
For background problem that triggered this problem, see [support#17565](https://support.isc.org/Ticket/Display.html?id=17565).kea2.3.1Francis DupontFrancis Duponthttps://gitlab.isc.org/isc-projects/kea/-/issues/2619update hooks_unittests in v4 and v6 to address #2548 review comments2023-07-17T13:58:23ZRazvan Becheriuupdate hooks_unittests in v4 and v6 to address #2548 review commentskea2.3.3Razvan BecheriuRazvan Becheriuhttps://gitlab.isc.org/isc-projects/kea/-/issues/2764Instrument lease manager for tracking lease changes2023-07-17T13:58:22ZMarcin SiodelskiInstrument lease manager for tracking lease changesIt is a ground work for the Free Lease Queue Allocator. The allocator must be aware of lease allocations and deletions to record the lease availability in the pool allocation state. An allocated lease should be removed from the state. A ...It is a ground work for the Free Lease Queue Allocator. The allocator must be aware of lease allocations and deletions to record the lease availability in the pool allocation state. An allocated lease should be removed from the state. A released or reclaimed lease should be added back to the pool allocation state. The LeaseMgr is the most reliable source of information about any lease changes because it handles both the allocations from the allocation engine and from the command channel. Therefore, this ticket instruments the LeaseMgr (for all backends) to install callbacks triggered when the leases are modified in the database. The FLQ allocator will use this mechanism to get notifications about such modifications. The mechanism is obviously optional and should not affect performance of the lease manager when the callbacks (and FLQ) are unused.
This mechanism is also going to be potentially useful for other applications. For instance, one can imagine having callbacks that record lease allocations to make them available to Stork.kea2.3.6Marcin SiodelskiMarcin Siodelskihttps://gitlab.isc.org/isc-projects/kea/-/issues/145feature request: ability to get lease count per pool2023-07-17T13:58:21ZGhost Userfeature request: ability to get lease count per pooltoday, when using the api command - "statistic-get-all"
we are getting a lease count at the subnet level,
i'm asking to get a count at the pool level.
thank you
#huge-sorrytoday, when using the api command - "statistic-get-all"
we are getting a lease count at the subnet level,
i'm asking to get a count at the pool level.
thank you
#huge-sorrykea2.3.8Razvan BecheriuRazvan Becheriuhttps://gitlab.isc.org/isc-projects/kea/-/issues/2891Handling optional columns in SELECT for PostgreSQL2023-06-15T13:51:46ZMarcin SiodelskiHandling optional columns in SELECT for PostgreSQLDuring the discussions about #2886 it was apparent that we need to design a way of making SELECT queries in PostgreSQL with excluding some columns. It is particularly important for the `lease6` table queries where we recently started add...During the discussions about #2886 it was apparent that we need to design a way of making SELECT queries in PostgreSQL with excluding some columns. It is particularly important for the `lease6` table queries where we recently started adding new columns (related to BLQ) that are excluded in some (many?) cases. The `PgSqlLease6Exchange` contains the list of constants designating the indexes of the columns in the SELECT statements. A temporary solution was to assign the highest indexes for optional columns (put them at the end), but it may be good for a single optional column. If you have two optional columns this solution has serious limitations. For example, to query the last optional column you need to query all preceding optional columns, even if they are not needed in the particular case.
The following thread in #2886 is relevant: https://gitlab.isc.org/isc-projects/kea/-/merge_requests/2016#note_377155
It includes some ideas how to address this problem.https://gitlab.isc.org/isc-projects/kea/-/issues/875Propose a better allocation strategy2023-04-06T11:13:40ZTomek MrugalskiPropose a better allocation strategyKea 1.6 and earlier always used the same allocation strategy: pick the first address/prefix, look into the DB whether it's available. If not move to the next address/prefix and try again. That works very well when the pool is mostly empt...Kea 1.6 and earlier always used the same allocation strategy: pick the first address/prefix, look into the DB whether it's available. If not move to the next address/prefix and try again. That works very well when the pool is mostly empty, but is very inefficient when the pool is mostly full as it requires many lookups.
Even with multi-threading, we would just do the same inefficient thing over and over again, just faster. The ultimate solution to this problem is to decrease the number of lookups.
The [performance proposals for 1.7](https://gitlab.isc.org/isc-private/kea/wikis/performance1.7) list several ideas what could be done about it:
- keep free leases in the DB. That way we could do a single SELECT to return the first free/expired lease.
- alternative proposal calls for lease cache. We could do a single query and return all free leases in a pool. This would work great for single single-threaded instance, but it's unclear how to maintain this cache consistency among multiple threads or even worse multiple Kea instances connected to the same DB.
- stored procedure to return empty lease. This should improve the DB round trip time, but at the end of the day we would just repeat the same inefficient algorithm: when the pool is mostly full, the stored procedure would take a lot of time to complete.
Related ticket: #1126.current-stable-2.4https://gitlab.isc.org/isc-projects/kea/-/issues/2445PostgreSQL schema support for lease limits2022-07-07T11:47:52ZAndrei Pavelandrei@isc.orgPostgreSQL schema support for lease limitsAdd the functionality to support lease limits to the PostgreSQL lease manager as outlined below:
* [x] Create lease count tables to track leases per client class and per subnet
* [x] Create the triggers to update lease counts upon lease...Add the functionality to support lease limits to the PostgreSQL lease manager as outlined below:
* [x] Create lease count tables to track leases per client class and per subnet
* [x] Create the triggers to update lease counts upon lease events: INSERT, UPDATE, DELETE
* [x] Create a function or procedure that compares all the configured lease limits relevant to a packet to all the current lease counts of the respective client classes and subnetskea2.2.0 - a new stable branchAndrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/2438MySQL schema support for lease limits2022-07-07T11:47:52ZAndrei Pavelandrei@isc.orgMySQL schema support for lease limitsAdd the functionality to support lease limits to the MySQL lease manager as outlined below:
* [x] Create lease count tables to track leases per client class and per subnet
* [x] Create the triggers to update lease counts upon lease even...Add the functionality to support lease limits to the MySQL lease manager as outlined below:
* [x] Create lease count tables to track leases per client class and per subnet
* [x] Create the triggers to update lease counts upon lease events: INSERT, UPDATE, DELETE
* [x] Create a function or procedure that compares all the configured lease limits relevant to a packet to all the current lease counts of the respective client classes and subnetskea2.1.7Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/2444LeaseMgr interface to limit checking2022-06-21T18:23:06ZAndrei Pavelandrei@isc.orgLeaseMgr interface to limit checkingAdd the necessary methods to the `LeaseMgr` base class and derived classes for checking if a lease limit has been reached.Add the necessary methods to the `LeaseMgr` base class and derived classes for checking if a lease limit has been reached.kea2.1.7Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/1086implement CqlLeaseMgr getLeases6SubnetId and getLeases62022-03-31T08:29:36ZRazvan Becheriuimplement CqlLeaseMgr getLeases6SubnetId and getLeases6missing implementation for getLeases6SubnetId and getLeases6 in CqlLeaseMgrmissing implementation for getLeases6SubnetId and getLeases6 in CqlLeaseMgroutstandingRazvan BecheriuRazvan Becheriuhttps://gitlab.isc.org/isc-projects/kea/-/issues/1078kea lease migration tool2022-02-18T08:00:21ZPeter Davieskea lease migration tool---
name: kea lease migration tool
RT: [support#15853](https://support.isc.org/Ticket/Display.html?id=15853)
There maybe some interest in customers who wish to migrate the lease database between different backends for a tool to assist th...---
name: kea lease migration tool
RT: [support#15853](https://support.isc.org/Ticket/Display.html?id=15853)
There maybe some interest in customers who wish to migrate the lease database between different backends for a tool to assist them in this.
Some extra notes:
* Alberto Garrido wrote a script for this: [here](https://gist.github.com/agrrd/287446084bab09a6628a7624f394a642)
* request (and some discussion) on [kea-users](https://lists.isc.org/pipermail/kea-users/2021-January/002977.html)kea2.1-backloghttps://gitlab.isc.org/isc-projects/kea/-/issues/2236add hwtype and hwaddr_source columns to memfile2022-01-18T13:59:04ZAndrei Pavelandrei@isc.orgadd hwtype and hwaddr_source columns to memfileRequired by #2038 and #2039.Required by #2038 and #2039.kea2.1.2Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/2257lease not loaded from memfile if user context has multiple key-value pairs2022-01-12T12:26:19ZAndrei Pavelandrei@isc.orglease not loaded from memfile if user context has multiple key-value pairsmemfile CSV:
```
address,hwaddr,client_id,valid_lifetime,expire,subnet_id,fqdn_fwd,fqdn_rev,hostname,state,user_context
10.0.0.1,ff:01:02:03:04:08,01:ff:01:02:03:04:08,7200,1641205200,1,0,0,,0,{"comment": "hello", "comment2": "do not rel...memfile CSV:
```
address,hwaddr,client_id,valid_lifetime,expire,subnet_id,fqdn_fwd,fqdn_rev,hostname,state,user_context
10.0.0.1,ff:01:02:03:04:08,01:ff:01:02:03:04:08,7200,1641205200,1,0,0,,0,{"comment": "hello", "comment2": "do not release"}
```
error:
```
ERROR DHCPSRV_MEMFILE_LEASE_LOAD_ROW_ERROR discarding row x, error: EOF read, one of ",}" expected in <string>:y:z
```
See `MemfileLeaseMgrTest.v4UserContext` and `MemfileLeaseMgrTest.v6UserContext` unit tests. They will require adjustments.https://gitlab.isc.org/isc-projects/kea/-/issues/608Lease file corrupted by comma in hostname2022-01-07T12:45:43ZGhost UserLease file corrupted by comma in hostname---
name: Lease file corrupted by comma in hostname
about: KEA 1.4 Lease file
---
**Describe the bug**
The KEA DHCP server that provides DHCP for our guest network couldn't be started (after being stopped) because the lease file was cor...---
name: Lease file corrupted by comma in hostname
about: KEA 1.4 Lease file
---
**Describe the bug**
The KEA DHCP server that provides DHCP for our guest network couldn't be started (after being stopped) because the lease file was corrupted.
After some digging we found the issue:
The lease file contained hostnames which had a comma in the hostname (please don't ask me why someone would do this). Since the leasefile contained over 100 records of hostnames with a comma, the KEA DHCP service wouldn't start because of a "corrupted lease file".
I've noticed that spaces in the hostname are converted inside the leasefile to "\032". However, comma's aren't converted, and break the leasefile, since its a CSV.
**To Reproduce**
Steps to reproduce the behavior:
1. lease an IP address to a host with a comma in the hostname
2. stop the kea service
3. start the kea service
4. look at the logging:
`2019-05-15 13:08:38.647 ERROR [kea-dhcp4.dhcpsrv/485] DHCPSRV_MEMFILE_LEASE_LOAD_ROW_ERROR discarding row 637, error: bad lexical cast: source type value could not be interpreted as target`
5. If more than 100 of these lines occur the KEA service won't start.
**Expected behavior**
Expected behavior is that comma's within the hostname (even though that is a bad practice) will be converted as well (just like space characters).
**Environment:**
- Kea version: KEA 1.4
- OS: Ubuntu 16.04 x64
**Additional Information**
Example hostname:
> HOST, NAME01kea1.7.6Thomas MarkwalderThomas Markwalderhttps://gitlab.isc.org/isc-projects/kea/-/issues/1230design for Incremental lease retrieval2021-03-26T11:45:17ZTomek Mrugalskidesign for Incremental lease retrievalThere's a need in Stork to be able to retrieve incremental lease updates. The goal here is to be able to get the state of all leases once during Stork start-up and then somehow be able to retrieve only incremental updates.
There are sev...There's a need in Stork to be able to retrieve incremental lease updates. The goal here is to be able to get the state of all leases once during Stork start-up and then somehow be able to retrieve only incremental updates.
There are several ways how this could potentially be implemented:
1. we could use HA backup server functionality - Stork would configure itself as backup server and would receive updates as they appear. The benefit with this approach is less signalling when there's little traffic. Another benefit is that the update would become visible almost immediately. The downside is that if Stork becomes unavailable for some time (being restarted, system reboots etc), it would potentially miss some updates.
2. an alternative apprach is to implement lease-get-all-since or lease-get-recent, a new command that would retrieve only leases updated after certain timestamp. We already have the info in lease DB, we would have to implement a new query to retrieve it. This has the advantage of always getting all the leases. The downside is that we'd have to implement more (this should work for all backends). Also, this has a potential to be more distruptive for Kea (or more precisely the disruption would be more concentrated in discrete event rather than spread out as in the proposal 1).
Need to handle deleted leases as well.kea1.9.6Marcin SiodelskiMarcin Siodelskihttps://gitlab.isc.org/isc-projects/kea/-/issues/1540remove the by hwaddr and clientid lease get method from the API2021-01-04T16:37:58ZFrancis Dupontremove the by hwaddr and clientid lease get method from the APIThree reasons:
- first there is a comment saying it should be removed
- the allocation engine is old enough now to confirm it will be never used with a probabity very close to 1
- the memory backend implementation is wrong (it does no...Three reasons:
- first there is a comment saying it should be removed
- the allocation engine is old enough now to confirm it will be never used with a probabity very close to 1
- the memory backend implementation is wrong (it does not copy it) and database backends do not implement itkea1.9.4Francis DupontFrancis Duponthttps://gitlab.isc.org/isc-projects/kea/-/issues/991Build some multi-threading unit tests for lease backends.2020-12-10T19:40:56ZFrancis DupontBuild some multi-threading unit tests for lease backends.kea1.9.3Razvan BecheriuRazvan Becheriuhttps://gitlab.isc.org/isc-projects/kea/-/issues/1415Create container holding free leases for lease pre-allocation feature2020-09-16T14:49:39ZMarcin SiodelskiCreate container holding free leases for lease pre-allocation featureThe following document: https://gitlab.isc.org/isc-projects/kea/-/wikis/designs/Backend-Assisted-Lease-Selection-design describes how we're going to deal with the leases pre-allocation to improve performance of allocation when the pools ...The following document: https://gitlab.isc.org/isc-projects/kea/-/wikis/designs/Backend-Assisted-Lease-Selection-design describes how we're going to deal with the leases pre-allocation to improve performance of allocation when the pools are nearly exhausted. The key aspect of this design is how to store the free leases. This issue is meant to come up with a PoC of the container for free leases that would meet (or at least get close to) the following requirements:
- it must be possible to pre-allocate only a part of the pool and allocate more on demand,
- the offered leases must be returned in "random" order,
- the lease reclamation process should return the leases into this container efficiently,
- reclaimed leases should go to the back of the queue for the pool which they belong to (in case they belong to a pool),
- the state should be stored within the container, i.e. externally I should only call "give me a new lease and this is my 4th attempt"
- the container must not be tightly coupled with the Memfile backend so as it can be re-used as a lease pre-allocation mechanism for other backendskea1.9.0Marcin SiodelskiMarcin Siodelskihttps://gitlab.isc.org/isc-projects/kea/-/issues/1225Error: invalid byte sequence for encoding "UTF8"2020-05-14T14:48:50ZALOK KUMAR SINGHError: invalid byte sequence for encoding "UTF8"See below logs in posgresql log files.....
```
ERROR: invalid byte sequence for encoding "UTF8": 0x93
STATEMENT: INSERT INTO lease4(address, hwaddr, client_id, valid_lifetime, expire, subnet_id, fqdn_fwd, fqdn_rev, hostname, state)...See below logs in posgresql log files.....
```
ERROR: invalid byte sequence for encoding "UTF8": 0x93
STATEMENT: INSERT INTO lease4(address, hwaddr, client_id, valid_lifetime, expire, subnet_id, fqdn_fwd, fqdn_rev, hostname, state) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
ERROR: invalid byte sequence for encoding "UTF8": 0x93
STATEMENT: INSERT INTO lease4(address, hwaddr, client_id, valid_lifetime, expire, subnet_id, fqdn_fwd, fqdn_rev, hostname, state) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
ERROR: invalid byte sequence for encoding "UTF8": 0x93
STATEMENT: INSERT INTO lease4(address, hwaddr, client_id, valid_lifetime, expire, subnet_id, fqdn_fwd, fqdn_rev, hostname, state) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
ERROR: invalid byte sequence for encoding "UTF8": 0x93
STATEMENT: INSERT INTO lease4(address, hwaddr, client_id, valid_lifetime, expire, subnet_id, fqdn_fwd, fqdn_rev, hostname, state) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
```https://gitlab.isc.org/isc-projects/kea/-/issues/994Leasequery design2020-04-01T11:33:54ZTomek MrugalskiLeasequery designWe need to implement leasequery mechanism defined in [RFC4388](https://tools.ietf.org/html/rfc4388) and [RFC5007](https://tools.ietf.org/html/rfc5007).
This is what people sometimes refer to as basic leasequery. This is something that's...We need to implement leasequery mechanism defined in [RFC4388](https://tools.ietf.org/html/rfc4388) and [RFC5007](https://tools.ietf.org/html/rfc5007).
This is what people sometimes refer to as basic leasequery. This is something that's relatively straightforward to do. It's a bit like lease4-get and lease6-get done over DHCP packet sent over UDP as normal DHCP packets. We have the backend capability in place. What we're missing is the ability to process leaseuqery packets.
There are three leasequery protocols:
- basic leasequery (RC4388, RFC5007) - a query about a single entry (who has address X, what address does client X have?) is sent over UDP, a response is a single packet.
- bulk leasequery (RFC6926 - v4, RFC5460 - v6). This is a TCP connection. There questions may return multiple answers (give me all leases connected via relay X), the answer is a series of DHCP datagrams sent over TCP. This connection is requestor initiated.
- active leasequery (RFC7724, RFC7653) - This is TCP based. The server provides a service which requestors can subscribe to. Once subscribed, they will receive any lease updates in nearly real-time. This is somewhat similar to what we have in HA.
While in the scope of 1.7.2 we want to only support basic leasequery, it is worth looks at bulk and active from the design perspective. There are customers who are potentially interested in bulk, so it's not far fetched to imagine that we'll be implementing it some time in the future.kea1.7.7Thomas MarkwalderThomas Markwalder