DDNS: addresses assigned from an arpa domain that is not configured can halt ddns processing
Kea Version tested: 2.4.0 with DHCPv4. Assumedly this same problem would exist in DHCPv6 but I didn't try that. The BIND version used in the test was 9.18.24, but I don't think it probably matters what version or brand of DNS server is used.
It has been discovered that it is possible that kea-dhcp-ddns can enter a state where no ddns updates are issued under certain circumstances. The circumstances required are only an intermittently unavailable DNS server, an address range in Kea that is not in the "reverse-ddns" portion of the DDNS configuration, a high rate of DHCP queries (I tested with 200 per second), and "ddns-update-on-renew": true in the kea-dhcp4 configuration. Below is the test scenario (first with a working version of the ddns configuration):
Kea configuration and command line
Command: kea-dhcp4 -c kea-dhcp4-test-ddns.json
kea-dhcp4-test-ddns.json
{
"Dhcp4": {
"interfaces-config": {
"interfaces": [
"ens256"
]
},
"lease-database": {
"type": "memfile",
"persist": false
},
"calculate-tee-times": true,
"valid-lifetime": 7200,
"ddns-generated-prefix": "myhost",
"ddns-qualifying-suffix": "example.org",
"ddns-replace-client-name": "always",
"ddns-send-updates": true,
"ddns-override-client-update": true,
"ddns-override-no-update": true,
"ddns-update-on-renew": true,
"dhcp-ddns": {
"enable-updates": true,
"max-queue-size": 1024,
"ncr-format": "JSON",
"ncr-protocol": "UDP",
"sender-ip": "0.0.0.0",
"sender-port": 0,
"server-ip": "127.0.0.1",
"server-port": 53001
},
"shared-networks": [
{
"name": "my-secret-lair-level-1",
"interface": "ens256",
"subnet4": [
{
"subnet": "10.1.2.0/24",
"id": 1,
"option-data": [
{
"name": "routers",
"data": "10.1.2.1"
}
],
"pools": [
{
"pool": "10.1.2.100-10.1.2.200"
}
]
},
{
"subnet": "172.16.0.0/24",
"id": 2,
"option-data": [
{
"name": "routers",
"data": "172.16.0.1"
}
],
"pools": [
{
"pool": "172.16.0.100-172.16.0.200"
}
]
}
]
}
],
"loggers": [
{
"name": "kea-dhcp4",
"severity": "DEBUG",
"debuglevel": 99,
"output_options": [
{
"output": "stdout"
}
]
}
]
}
}BIND configuration and command line
Command: named -4 -g -c /tmp/named.conf
named.conf
options {
directory "/tmp";
recursion no;
allow-update { any;};
dnssec-validation no;
};
zone "2.1.10.in-addr.arpa" {
type primary;
file "/tmp/2.1.10.in-addr.arpa";
};
zone "0.16.172.in-addr.arpa" {
type primary;
file "/tmp/0.16.172.in-addr.arpa";
};
zone "example.org" {
type primary;
file "/tmp/example.org";
};example.org
$ORIGIN .
$TTL 86399 ; 23 hours 59 minutes 59 seconds
example.org IN SOA ns1.example.org. hostmaster.example.org. (
1 ; serial
43200 ; refresh (12 hours)
900 ; retry (15 minutes)
1814400 ; expire (3 weeks)
7200 ; minimum (2 hours)
)
NS ns1.example.org.
ns1.example.org A 192.168.20.1142.1.10.in-addr.arpa
$ORIGIN .
$TTL 86400 ; 1 day
2.1.10.in-addr.arpa IN SOA 2.1.10.IN-ADDR.ARPA. . (
1 ; serial
30800 ; refresh (8 hours 33 minutes 20 seconds)
7200 ; retry (2 hours)
604800 ; expire (1 week)
300 ; minimum (5 minutes)
)
NS ns1.example.org.0.16.172.in-addr.arpa
$ORIGIN .
$TTL 86400 ; 1 day
0.16.172.in-addr.arpa IN SOA 0.16.172.in-addr.arpa. . (
1 ; serial
30800 ; refresh (8 hours 33 minutes 20 seconds)
7200 ; retry (2 hours)
604800 ; expire (1 week)
300 ; minimum (5 minutes)
)
NS ns1.example.org.Working DDNS configuration and command line
Command: kea-dhcp-ddns -c kea-dhcp-ddns-test-ddns.json
kea-dhcp-ddns-test-ddns.json
{
"DhcpDdns": {
"dns-server-timeout": 40000,
"forward-ddns": {
"ddns-domains": [
{
"dns-servers": [
{
"ip-address": "192.168.20.114",
"port": 53
}
],
"name": "example.org."
}
]
},
"reverse-ddns": {
"ddns-domains": [
{
"dns-servers": [
{
"ip-address": "192.168.20.114",
"port": 53
}
],
"name": "2.1.10.in-addr.arpa."
},
{
"dns-servers": [
{
"ip-address": "192.168.20.114",
"port": 53
}
],
"name": "0.16.172.in-addr.arpa."
}
]
},
"ip-address": "127.0.0.1",
"ncr-format": "JSON",
"ncr-protocol": "UDP",
"port": 53001,
"loggers": [
{
"severity": "DEBUG",
"debuglevel": 99,
"name": "kea-dhcp-ddns",
"output_options": [
{
"output": "stdout"
}
]
}
]
}
}non-Working DDNS configuration and command line
Command: kea-dhcp-ddns -c kea-dhcp-ddns-test-ddns.json
kea-dhcp-ddns-test-ddns.json
{
"DhcpDdns": {
"dns-server-timeout": 40000,
"forward-ddns": {
"ddns-domains": [
{
"dns-servers": [
{
"ip-address": "192.168.20.114",
"port": 53
}
],
"name": "example.org."
}
]
},
"reverse-ddns": {
"ddns-domains": [
{
"dns-servers": [
{
"ip-address": "192.168.20.114",
"port": 53
}
],
"name": "2.1.10.in-addr.arpa."
}
//,
// {
// "dns-servers": [
// {
// "ip-address": "192.168.20.114",
// "port": 53
// }
// ],
// "name": "0.16.172.in-addr.arpa."
// }
]
},
"ip-address": "127.0.0.1",
"ncr-format": "JSON",
"ncr-protocol": "UDP",
"port": 53001,
"loggers": [
{
"severity": "DEBUG",
"debuglevel": 99,
"name": "kea-dhcp-ddns",
"output_options": [
{
"output": "stdout"
}
]
}
]
}
}Perfdhcp was used to create the traffic for this test: sudo perfdhcp -4 -r 200 -p 1800 -l ens256 -R 200
BIND will, for some reason, stop responding intermittently during the test. The reason for that is not important for this issue. This was originally reported by a customer using some kind of off premise DNS servers that would intermittently be unavailable due to network issues. If all subnets are configured in the DDNS configuration, then DDNS will not become unresponsive when BIND becomes unresponsive.
This message might appear while BIND is unresponsive:
DHCP_DDNS_AT_MAX_TRANSACTIONS application has 1024 queued requests but has reached maximum number of 32 concurrent transactions
but DDNS will recover once BIND recovers.
Using the "non-Working DDNS configuration and command line", the DDNS server cannot recover and is unavailable for the remainder of the test.
The kea-dhcp-ddns service must be restarted before it will respond again.
I also tested this with example.org removed from the ddns configuration. kea-dhcp-ddns did not suffer a stop in processing with that zone removed. It appears to only be in the case of a missing .arpa zone.
When the `kea-dhcp-ddns` stops responding, it is during one of these failures to match reverse DNS zone
2024-03-19 16:56:18.347 WARN [kea-dhcp-ddns.dhcp-to-d2/1479.281473066429376] DHCP_DDNS_NO_MATCH No DNS servers match FQDN 149.0.16.172.in-addr.arpa.
2024-03-19 16:56:18.347 ERROR [kea-dhcp-ddns.dhcp-to-d2/1479.281473066429376] DHCP_DDNS_NO_REV_MATCH_ERROR Request ID 000101285974D2A2411A8BCED2CF77E9E97AD8582814F422CD88FD27E2B37B26969C5F: the configured list of reverse DDNS domains does not contain a match for: Type: 1 (CHG_REMOVE)
Forward Change: yes
Reverse Change: yes
FQDN: [myhost-172-16-0-149.example.org.]
IP Address: [172.16.0.149]
DHCID: [000101285974D2A2411A8BCED2CF77E9E97AD8582814F422CD88FD27E2B37B26969C5F]
Lease Expires On: 20240319173614
Lease Length: 2400
Conflict Resolution: yes
The request has been discarded.No further logs are emitted by kea-dhcp-ddns until the process is restarted.
`kea-dhcp4` does not appear to realize anything is amiss
2024-03-19 16:58:41.932 DEBUG [kea-dhcp4.dhcpsrv/1487.281473627656064] DHCPSRV_QUEUE_NCR [hwtype=1 00:0c:01:02:03:23], cid=[01:00:0c:01:02:03:23]: Name change request to remove DNS entry queued: Type: 1 (CHG_REMOVE)
Forward Change: yes
Reverse Change: yes
FQDN: [myhost-10-1-2-131.example.org.]
IP Address: [10.1.2.131]
DHCID: [000101EC31CD9751563A5FD3586A0940AEDE3871AA5D6D952E92D3D5A21E173B5F146C]
Lease Expires On: 20240319173840
Lease Length: 2400
Conflict Resolution: yes
2024-03-19 16:58:41.932 DEBUG [kea-dhcp4.dhcpsrv/1487.281473669656592] DHCPSRV_DHCP_DDNS_NCR_SENT NameChangeRequest sent to kea-dhcp-ddns: Type: 1 (CHG_REMOVE)
Forward Change: yes
Reverse Change: yes
FQDN: [myhost-10-1-2-131.example.org.]
IP Address: [10.1.2.131]
DHCID: [000101EC31CD9751563A5FD3586A0940AEDE3871AA5D6D952E92D3D5A21E173B5F146C]
Lease Expires On: 20240319173840
Lease Length: 2400
Conflict Resolution: yes
2024-03-19 16:58:41.932 DEBUG [kea-dhcp4.dhcpsrv/1487.281473627656064] DHCPSRV_QUEUE_NCR [hwtype=1 00:0c:01:02:03:23], cid=[01:00:0c:01:02:03:23]: Name change request to add DNS entry queued: Type: 0 (CHG_ADD)
Forward Change: yes
Reverse Change: yes
FQDN: [myhost-10-1-2-131.example.org.]
IP Address: [10.1.2.131]
DHCID: [000101EC31CD9751563A5FD3586A0940AEDE3871AA5D6D952E92D3D5A21E173B5F146C]
Lease Expires On: 20240319173841
Lease Length: 2400
Conflict Resolution: yes
2024-03-19 16:58:41.932 DEBUG [kea-dhcp4.dhcpsrv/1487.281473669656592] DHCPSRV_DHCP_DDNS_NCR_SENT NameChangeRequest sent to kea-dhcp-ddns: Type: 0 (CHG_ADD)
Forward Change: yes
Reverse Change: yes
FQDN: [myhost-10-1-2-131.example.org.]
IP Address: [10.1.2.131]
DHCID: [000101EC31CD9751563A5FD3586A0940AEDE3871AA5D6D952E92D3D5A21E173B5F146C]
Lease Expires On: 20240319173841
Lease Length: 2400
Conflict Resolution: yesas the log messages appear the same whether kea-dhcp-ddns is doing anything or not.