Kea 1.6.1 and 1.7.2, mysql configuration backend, clients not always getting a gateway
We have paid for the premier hooks, so were running kea with the mysql configuration backend for hosts, subnets and leases. This is a dhcp4 only environment.
Our environment consists of around 300 subnets
We have a test host with a out of band management device that is set to use our new kea server via dhcp helper address.
If we update kea, via posting a subnet config to the agent controller, for this management network, we can restart the out-of-band management device and it gets an address and a gateway
If we add a few more subnets, again, posting a subnet config to the agent controller, we can restart the OOB device and it gets an IP address and a gateway
This is where things get weird.
If we add all 300+ subnets, and restart the OOB device, it does not get a gateway. It gets a leased ip address, but no gateway.
Our packet captures during these events shows our OOB device requesting a gateway, but kea never sends one:
20:54:46.558258 IP (tos 0x0, ttl 60, id 0, offset 0, flags [DF], proto UDP (17), length 328)
10.17.232.2.bootps > dhcp-srv-1.example.net.bootps: [udp sum ok] BOOTP/DHCP, Request from d0:94:66:22:8d:32 (oui Unknown), length 300, hops 1, xid 0xfd9d1412, Flags [none] (0x0000)
Gateway-IP 10.17.232.2
Client-Ethernet-Address d0:94:66:22:8d:32 (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Discover
Client-ID Option 61, length 7: ether d0:94:66:22:8d:32
MSZ Option 57, length 2: 576
Parameter-Request Option 55, length 10:
Subnet-Mask, Default-Gateway, Domain-Name-Server, Hostname
Domain-Name, BR, NTP, Vendor-Option
Classless-Static-Route, Classless-Static-Route-Microsoft
Vendor-Class Option 60, length 5: "iDRAC"
Hostname Option 12, length 13: "idrac-abc1234"
END Option 255, length 0
PAD Option 0, length 0, occurs 9
20:54:46.574260 IP (tos 0x0, ttl 64, id 24939, offset 0, flags [DF], proto UDP (17), length 314)
dhcp-srv-1.example.net.bootps > 10.17.232.2.bootps: [bad udp cksum 0x1f28 -> 0xbe43!] BOOTP/DHCP, Reply, length 286, hops 1, xid 0xfd9d1412, Flags [none] (0x0000)
Your-IP 10.17.232.6
Gateway-IP 10.17.232.2
Client-Ethernet-Address d0:94:66:22:8d:32 (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Offer
Subnet-Mask Option 1, length 4: 255.255.248.0
Hostname Option 12, length 13: "idrac-abc1234"
Lease-Time Option 51, length 4: 172800
Server-ID Option 54, length 4: dhcp-srv-1.example.net
Client-ID Option 61, length 7: ether d0:94:66:22:8d:32
END Option 255, length 0If we re-add that OOB subnet though, the OOB device will get a gateway:
21:19:27.005872 IP (tos 0x0, ttl 60, id 0, offset 0, flags [DF], proto UDP (17), length 331)
10.17.232.3.bootps > dhcp-srv-1.example.net.bootps: [udp sum ok] BOOTP/DHCP, Request from d0:94:66:22:8d:32 (oui Unknown), length 303, hops 1, xid 0x4eafdb02, Flags [none] (0x0000)
Gateway-IP 10.17.232.3
Client-Ethernet-Address d0:94:66:22:8d:32 (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Request
Client-ID Option 61, length 7: ether d0:94:66:22:8d:32
Requested-IP Option 50, length 4: 10.17.232.6
Server-ID Option 54, length 4: dhcp-srv-1.example.net
MSZ Option 57, length 2: 576
Parameter-Request Option 55, length 10:
Subnet-Mask, Default-Gateway, Domain-Name-Server, Hostname
Domain-Name, BR, NTP, Vendor-Option
Classless-Static-Route, Classless-Static-Route-Microsoft
Vendor-Class Option 60, length 5: "iDRAC"
Hostname Option 12, length 13: "idrac-abc1234"
END Option 255, length 0
21:19:27.010985 IP (tos 0x0, ttl 64, id 5150, offset 0, flags [DF], proto UDP (17), length 364)
dhcp-srv-1.example.net.bootps > 10.17.232.3.bootps: [bad udp cksum 0x1f5b -> 0x9adb!] BOOTP/DHCP, Reply, length 336, hops 1, xid 0x4eafdb02, Flags [none] (0x0000)
Your-IP 10.17.232.6
Gateway-IP 10.17.232.3
Client-Ethernet-Address d0:94:66:22:8d:32 (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Offer
Subnet-Mask Option 1, length 4: 255.255.248.0
Default-Gateway Option 3, length 4: 10.17.232.1
Domain-Name-Server Option 6, length 12: ns1.example.net,ns2.example.net,ns3.example.net
Hostname Option 12, length 13: "idrac-abc1234"
Domain-Name Option 15, length 14: "oob.example.net"
NTP Option 42, length 12: ns-1.example.net,ns-2.example.net,ns-3.example.net
Lease-Time Option 51, length 4: 172800
Server-ID Option 54, length 4: dhcp-srv-1.example.net
Client-ID Option 61, length 7: ether d0:94:66:22:8d:32
END Option 255, length 0If we then re-add all 300 subnets, our OOB device will not get a gateway.
We would really like to understand why this happens.
For the last bit of info, this is our OOB subnet config:
[
{
"arguments": {
"count": 1,
"subnets": [
{
"4o6-interface": "",
"4o6-interface-id": "",
"4o6-subnet": "",
"id": 506,
"metadata": {
"server-tags": [
"all"
]
},
"option-data": [
{
"always-send": false,
"code": 3,
"csv-format": true,
"data": "10.17.232.1",
"name": "routers",
"space": "dhcp4"
},
{
"always-send": false,
"code": 15,
"csv-format": true,
"data": "oob.example.net",
"name": "domain-name",
"space": "dhcp4"
},
{
"always-send": false,
"code": 119,
"csv-format": true,
"data": "example.net",
"name": "domain-search",
"space": "dhcp4"
},
{
"always-send": false,
"code": 42,
"csv-format": true,
"data": "10.1.43.10, 10.1.45.10, 10.1.42.10",
"name": "ntp-servers",
"space": "dhcp4"
},
{
"always-send": false,
"code": 6,
"csv-format": true,
"data": "10.1.241.220, 10.1.241.221, 10.1.241.222",
"name": "domain-name-servers",
"space": "dhcp4"
}
],
"pools": [
{
"option-data": [],
"pool": "10.17.232.6-10.17.239.254"
}
],
"relay": {
"ip-addresses": []
},
"shared-network-name": null,
"subnet": "10.17.232.0/21",
"user-context": {
"location": "oob"
}
}
]
},
"result": 0,
"text": "IPv4 subnet 10.17.232.0/21 found."
}
]All of our subnets use the same template.