Assigning subnet ID values in an arbitrary order breaks global reservation logic via client-classes in kea-dhcp4
Describe the bug
Running stable Kea DHCPv4 daemon 1.8.2, configured with multiple subnets and client reservations there seem to work only if subnet "id" values are set in some undocumented order (likely monotonically-increasing one), and breaks if they are set to integers ordered in any other way.
For example, if first subnet in the config file has "id": 2
and second one has "id": 1
, Kea will fail to allocate addresses to clients, logging invalid subnet IDs (zeroes) and warnings like these from allocation engine:
WARN kea-dhcp4.alloc-engine ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0xbe907c24: failed to allocate an IPv4 address after 0 attempt(s)
WARN kea-dhcp4.alloc-engine ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0xbe907c24: Failed to allocate an IPv4 address for client with classes: ALL, VENDOR_CLASS_udhcp 1.32.1, 10.123.1.0-10.123.1.255, KNOWN
To Reproduce
Steps to reproduce the behavior:
-
Put following kea-dhcp4 configuration to e.g. kea.json file:
{"Dhcp4": { "loggers": [{"name": "kea-dhcp4", "severity": "DEBUG", "debuglevel": 99, "output_options": [{"output": "stdout"}]}], "lease-database": {"type": "memfile", "persist": false}, "reservation-mode": "global", "host-reservation-identifiers": ["hw-address"], "reservations": [{ "hw-address": "52:54:00:12:34:64", "client-classes": ["10.123.1.0-10.123.1.255"] }], "client-classes": [ {"name": "10.123.1.0-10.123.1.255"}, {"name": "10.50.1.0-10.50.1.255"}, {"name": "10.50.2.0-10.50.2.255"} ], "interfaces-config": {"interfaces": ["kea"]}, "subnet4": [ { "subnet": "10.123.0.0/16", "id": 2, "pools": [{"pool": "10.123.1.0/24", "client-class": "10.123.1.0-10.123.1.255"}], "interface": "kea" }, { "subnet": "10.50.0.0/16", "id": 1, "pools": [ {"pool": "10.50.1.0/24", "client-class": "10.50.1.0-10.50.1.255"}, {"pool": "10.50.2.0/24", "client-class": "10.50.2.0-10.50.2.255"} ], "interface": "kea" } ] }}
-
Adjust "kea" network interface name and IP/MAC address(-es) to something appropriate for the system, or create/tweak interfaces via ip link/addr commands on the local system. Note the "hw-address" value used in client reservation - it must match testing client's hw address.
-
Start kea-dhcp4 daemon with this configuration:
mkdir -p /tmp/kea ; KEA_PIDFILE_DIR=/tmp/kea KEA_LOCKFILE_DIR=/tmp/kea kea-dhcp4 -c kea.json
-
Try to request DHCP address from this daemon (using client with matching hw-address):
ip link show kea ; busybox udhcpc -t1 -fnqi kea
-
Observe that this client fails to get IP address via DHCP from Kea, despite matching reserved hw address:
# ip link show kea ; busybox udhcpc -t1 -fnqi kea 3: kea@ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether 52:54:00:12:34:64 brd ff:ff:ff:ff:ff:ff udhcpc: started, v1.32.1 udhcpc: sending discover udhcpc: no lease, failing
Relevant snippet from Kea logging (configured to be very verbose in that kea.json):
2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.packets/2873.140244699678592] DHCP4_BUFFER_RECEIVED received buffer from 0.0.0.0:68 to 255.255.255.255:67 over interface kea 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.options/2873.140244699678592] DHCP4_BUFFER_UNPACK parsing buffer received from 0.0.0.0 to 255.255.255.255 over interface kea 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.dhcpsrv/2873.140244699678592] DHCPSRV_CFGMGR_SUBNET4_IFACE selected subnet 10.50.0.0/16 for packet received over interface kea 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.packets/2873.140244699678592] DHCP4_SUBNET_SELECTED [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: the subnet with ID 1 was selected for client assignments 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.packets/2873.140244699678592] DHCP4_SUBNET_DATA [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: the selected subnet details: 10.50.0.0/16 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.packets/2873.140244699678592] DHCP4_PACKET_RECEIVED [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: DHCPDISCOVER (type 1) received from 0.0.0.0 to 255.255.255.255 on interface kea 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.packets/2873.140244699678592] DHCP4_QUERY_DATA [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09, packet details: local_address=255.255.255.255:67, remote_address=0.0.0.0:68, msg_type=DHCPDISCOVER (1), transid=0x8492bb09, options: type=053, len=001: 1 (uint8) type=055, len=007: 1(uint8) 3(uint8) 6(uint8) 12(uint8) 15(uint8) 28(uint8) 42(uint8) type=057, len=002: 576 (uint16) type=060, len=012: "udhcp 1.32.1" (string) type=061, len=007: 01:52:54:00:12:34:64 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.dhcpsrv/2873.140244699678592] DHCPSRV_CFGMGR_SUBNET4_IFACE selected subnet 10.50.0.0/16 for packet received over interface kea 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.packets/2873.140244699678592] DHCP4_SUBNET_SELECTED [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: the subnet with ID 1 was selected for client assignments 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.packets/2873.140244699678592] DHCP4_SUBNET_DATA [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: the selected subnet details: 10.50.0.0/16 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.hosts/2873.140244699678592] HOSTS_CFG_GET_ONE_SUBNET_ID_IDENTIFIER get one host with IPv4 reservation for subnet id 0, identified by hwaddr=525400123464 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.hosts/2873.140244699678592] HOSTS_CFG_GET_ALL_IDENTIFIER get all hosts with reservations using identifier: hwaddr=525400123464 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.hosts/2873.140244699678592] HOSTS_CFG_GET_ALL_IDENTIFIER_HOST using identifier: hwaddr=525400123464, found host: hwaddr=525400123464 ipv4_subnet_id=0 hostname=(empty) ipv4_reservation=(no) siaddr=(no) sname=(empty) file=(empty) key=(empty) ipv6_reservations=(none) dhcp4_class0=10.123.1.0-10.123.1.255 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.hosts/2873.140244699678592] HOSTS_CFG_GET_ALL_IDENTIFIER_COUNT using identifier hwaddr=525400123464, found 1 host(s) 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.hosts/2873.140244699678592] HOSTS_CFG_GET_ONE_SUBNET_ID_IDENTIFIER_HOST using subnet id 0 and identifier hwaddr=525400123464, found host: hwaddr=525400123464 ipv4_subnet_id=0 hostname=(empty) ipv4_reservation=(no) siaddr=(no) sname=(empty) file=(empty) key=(empty) ipv6_reservations=(none) dhcp4_class0=10.123.1.0-10.123.1.255 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.dhcp4/2873.140244699678592] DHCP4_CLASS_ASSIGNED [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: client packet has been assigned to the following class(es): KNOWN 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.dhcp4/2873.140244699678592] DHCP4_CLASS_ASSIGNED [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: client packet has been assigned to the following class(es): ALL, VENDOR_CLASS_udhcp 1.32.1, 10.123.1.0-10.123.1.255, KNOWN 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.ddns/2873.140244699678592] DHCP4_CLIENT_HOSTNAME_PROCESS [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: processing client's Hostname option 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.dhcpsrv/2873.140244699678592] DHCPSRV_MEMFILE_GET_CLIENTID obtaining IPv4 leases for client ID 01:52:54:00:12:34:64 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.dhcpsrv/2873.140244699678592] DHCPSRV_MEMFILE_GET_HWADDR obtaining IPv4 leases for hardware address hwtype=1 52:54:00:12:34:64 2021-03-14 11:21:34.061 DEBUG [kea-dhcp4.alloc-engine/2873.140244699678592] ALLOC_ENGINE_V4_OFFER_NEW_LEASE allocation engine will try to offer new lease to the client [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09 2021-03-14 11:21:34.062 WARN [kea-dhcp4.alloc-engine/2873.140244699678592] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: failed to allocate an IPv4 address after 0 attempt(s) 2021-03-14 11:21:34.062 WARN [kea-dhcp4.alloc-engine/2873.140244699678592] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: Failed to allocate an IPv4 address for client with classes: ALL, VENDOR_CLASS_udhcp 1.32.1, 10.123.1.0-10.123.1.255, KNOWN 2021-03-14 11:21:34.062 DEBUG [kea-dhcp4.bad-packets/2873.140244699678592] DHCP4_PACKET_NAK_0003 [hwtype=1 52:54:00:12:34:64], cid=[01:52:54:00:12:34:64], tid=0x8492bb09: failed to advertise a lease, client sent ciaddr 0.0.0.0, requested-ip-address (no address)
Aside from WARN messages mentioned in the description at the top, also note discrepancy in "subnet id" values logged here:
DHCP4_SUBNET_SELECTED ... the subnet with ID 1 was selected for client assignments DHCP4_SUBNET_DATA ... the selected subnet details: 10.50.0.0/16 HOSTS_CFG_GET_ONE_SUBNET_ID_IDENTIFIER get one host with IPv4 reservation for subnet id 0, ...
And
failed to allocate an IPv4 address after 0 attempt(s)
suggests either off-by-one error in attempt counting or something strange happening as well. -
Comment-out, remove or reorder "id" values in "subnet4" configuration section, restart kea-dhcp4 daemon and re-run the test.
Observe that everything works with "id" values missing or assigned in increasing order, confirming that issue is indeed related to these, as nothing else was changed there.
Expected behavior
Same as documented in Kea ARM "8.2.6. IPv4 Subnet Identifier" section - assigning any arbitrary positive integer values in 32-bit range as subnet identifiers using some internal logic works without any issues.
Using specific configuration above, client with matching hw-addr should be assigned to IP address from id=2 subnet, which is the first one in "subnet4" section there, even if configured subnet "id" values are not ordered in some specific way.
Environment:
- Kea version: current stable 1.8.2.
- OS: current Arch Linux (rolling) in a VM.
Additional Information
This issue can break any Kea automated configurations where subnets can come and go, but must maintain their ID values to match persistent leasedb info. Workaround for the issue seem to be always strictly sorting subnets in configuration by their "id" values - not doing that will result in DHCP service failure.
If this order is indeed critical for Kea operation and not a bug, I propose that it should be both documented and enforced by Kea configuration parser (failing validation if order is incorrect).
Though it does look more like a bug, likely created by exposing internally-used "id" values without considering implicit restrictions on these in the existing code.
I did not test this behavior without client reservations (as these are important to my use-case), so it is possible that something else fails if this id-values-ordering is not strict in the daemon configuration.
Contacting you
Should be able to monitor responses here.