Skip to content
GitLab
Closed
Issue created by Wlodzimierz Wencel@wlodekMaintainer

kea-netconf crashes when user attempt to update running datastore

  1. uploading yang configuration to 'startup' datastore (this config was previously uploaded to running datastore, but since kea wasn't running it wasn't applied):
<?xml version="1.0" ?>
<config xmlns="urn:ietf:params:xml:ns:yang:kea-dhcp4-server">
	<interfaces-config>
		<interfaces>enp0s9</interfaces>
	</interfaces-config>
	<client-classes/>
	<expired-leases-processing/>
	<option-data-list>
		<option-data>
			<code>2</code>
			<space>dhcp4</space>
			<data>55</data>
			<name>time-offset</name>
		</option-data>
	</option-data-list>
	<control-socket>
		<socket-name>/home/wlodek/installed/git/var/kea/control_socket</socket-name>
		<socket-type>unix</socket-type>
	</control-socket>
	<lease-database>
		<database-type>memfile</database-type>
		<name>/home/wlodek/installed/git/var/kea/kea-leases4.csv</name>
	</lease-database>
	<subnet4>
		<subnet4>
			<id>1</id>
			<subnet>10.0.0.0/24</subnet>
			<valid-lifetime>4000</valid-lifetime>
			<renew-timer>1000</renew-timer>
			<rebind-timer>2000</rebind-timer>
			<interface>enp0s9</interface>
			<option-data-list/>
			<relay>
				<ip-addresses>192.12.11.4</ip-addresses>
			</relay>
			<pools>
				<pool>
					<start-address>10.0.0.50</start-address>
					<end-address>10.0.0.54</end-address>
					<option-data-list/>
				</pool>
			</pools>
		</subnet4>
	</subnet4>
</config>
  1. Starting Kea with simple config:
{"Dhcp4":
 {"control-socket":
  {"socket-type":"unix",
   "socket-name":"/home/wlodek/installed/git/var/kea/control_socket"}
 },
 "Logging":
  {"loggers":[
   {"name":"kea-dhcp4",
    "output_options":[
    {"output":"/home/wlodek/installed/git/var/kea/kea.log"}],
     "debuglevel":99,
     "severity":"DEBUG"}]
  }
 }
  1. starting kea-netconf with:
{
    "Netconf":
    {
        "managed-servers": {
            "dhcp4": {
                "model": "kea-dhcp4-server",
                "comment": "DHCP4 server",
        "boot-update": true,
        "subscribe-changes": true,
        "validate-changes": true,
                "control-socket": {
                    "socket-name": "/home/wlodek/installed/git/var/kea/control_socket",
                    "socket-type": "unix"
                }
            }
        }
    },
    "Logging":
    {
        "loggers": [
        {
            "name": "kea-netconf",
            "output_options": [
                {
                    "output": ""
                }
            ],
            "severity": "DEBUG",
            "debuglevel":99
        }
        ]
    }
}

Starts successfully:

wlodek@debian9-64-2:~/installed$ sudo git/sbin/kea-netconf -c kea-netconf.conf -d 
2018-11-07 14:04:08.956 INFO  [kea-netconf.dctl/29229] DCTL_STARTING Netconf starting, pid: 29229, version: 1.4.0-git
2018-11-07 14:04:08.956 DEBUG [kea-netconf.dctl/29229] DCTL_INIT_PROCESS Netconf initializing the application
2018-11-07 14:04:08.957 DEBUG [kea-netconf.dctl/29229] DCTL_STANDALONE Netconf skipping message queue, running standalone
2018-11-07 14:04:08.957 DEBUG [kea-netconf.dctl/29229] DCTL_CONFIG_START parsing new configuration: { "managed-servers": { "dhcp4": { "boot-update": true, "control-socket": { "socket-name": "/home/wlodek/installed/git/var/kea/control_socket", "socket-type": "unix" }, "model": "kea-dhcp4-server", "subscribe-changes": true, "user-context": { "comment": "DHCP4 server" }, "validate-changes": true } } }
2018-11-07 14:04:08.958 INFO  [kea-netconf.dctl/29229] DCTL_CONFIG_COMPLETE server has completed configuration: dhcp4, 0 lib(s):
  1. at this point Kea is being reconfigured and logs (and it's able to assign addresses from 10.0.0.0/24 pool)
2018-11-07 14:04:08.995 INFO  [kea-dhcp4.dhcpsrv/29227] DHCPSRV_CFGMGR_SOCKET_TYPE_SELECT using socket type raw
2018-11-07 14:04:08.995 INFO  [kea-dhcp4.dhcpsrv/29227] DHCPSRV_CFGMGR_ADD_IFACE listening on interface enp0s9
2018-11-07 14:04:08.996 INFO  [kea-dhcp4.dhcpsrv/29227] DHCPSRV_CFGMGR_NEW_SUBNET4 a new subnet has been added to configuration: 10.0.0.0/24 with params: t1=1000, t2=2000, valid-lifetime=4000
2018-11-07 14:04:08.997 INFO  [kea-dhcp4.dhcp4/29227] DHCP4_CONFIG_COMPLETE DHCPv4 server has completed configuration: added IPv4 subnets: 1; DDNS: disabled
2018-11-07 14:04:08.999 INFO  [kea-dhcp4.dhcpsrv/29227] DHCPSRV_MEMFILE_DB opening memory file lease database: lfc-interval=3600 max-reconnect-tries=0 name=/home/wlodek/installed/git/var/kea/kea-leases4.csv persist=true readonly=false reconnect-wait-time=0 tcp-keepalive=0 tcp-nodelay=true type=memfile universe=4
  1. Upload new config:
<?xml version="1.0" ?>
<config xmlns="urn:ietf:params:xml:ns:yang:kea-dhcp4-server">
	<interfaces-config>
		<interfaces>enp0s9</interfaces>
	</interfaces-config>
	<client-classes/>
	<expired-leases-processing/>
	<option-data-list>
		<option-data>
			<code>2</code>
			<space>dhcp4</space>
			<data>55</data>
			<name>time-offset</name>
		</option-data>
	</option-data-list>
	<control-socket>
		<socket-name>/home/wlodek/installed/git/var/kea/control_socket</socket-name>
		<socket-type>unix</socket-type>
	</control-socket>
	<lease-database>
		<database-type>memfile</database-type>
		<name>/home/wlodek/installed/git/var/kea/kea-leases4.csv</name>
	</lease-database>
	<subnet4>
		<subnet4>
			<id>1</id>
			<subnet>192.168.51.0/24</subnet>
			<valid-lifetime>4000</valid-lifetime>
			<renew-timer>1000</renew-timer>
			<rebind-timer>2000</rebind-timer>
			<interface>enp0s9</interface>
			<option-data-list/>
			<relay>
				<ip-addresses>192.12.11.4</ip-addresses>
			</relay>
			<pools>
				<pool>
					<start-address>192.168.51.50</start-address>
					<end-address>192.168.51.54</end-address>
					<option-data-list/>
				</pool>
			</pools>
		</subnet4>
	</subnet4>
</config>
  1. Attempt to reconfigure Kea using 'running' datastore:
wlodek@debian9-64-2:~/installed$ sudo sysrepocfg -l 4 -d running -f xml -i /home/wlodek/installed/git/etc/kea/yang.xml kea-dhcp4-server
[DBG] (sr_log_stderr:141) Setting log level for stderr logs to 4.
[DBG] (sr_connect:380) Connecting to Sysrepo Engine.
[DBG] (cl_socket_connect:376) Connecting to socket=/tmp/sysrepod.sock
[INF] (sr_connect:435) Connected to daemon Sysrepo Engine at socket=/tmp/sysrepod.sock
[DBG] (cl_version_verify:431) Sending version-verify request.
[DBG] (cl_version_verify:442) version-verify request sent, waiting for response.
[DBG] (cl_version_verify:454) version-verify response received, processing.
[DBG] (cl_request_process:493) Sending session-start request.
[DBG] (cl_request_process:506) session-start request sent, waiting for response.
[DBG] (cl_request_process:519) session-start response received, processing.
[DBG] (cl_request_process:493) Sending check-enabled-running request.
[DBG] (cl_request_process:506) check-enabled-running request sent, waiting for response.
[DBG] (cl_request_process:519) check-enabled-running response received, processing.
[DBG] (srcfg_ly_init:189) Loading module schema: '/opt/sysrepo-0.7.5/build/repository/yang/kea-dhcp4-server@2018-09-14.yang'.
[DBG] (cl_request_process:493) Sending discard-changes request.
[DBG] (cl_request_process:506) discard-changes request sent, waiting for response.
[DBG] (cl_request_process:519) discard-changes response received, processing.
[DBG] (cl_request_process:493) Sending get-items request.
[DBG] (cl_request_process:506) get-items request sent, waiting for response.
[DBG] (cl_request_process:519) get-items response received, processing.
[DBG] (cl_request_process:493) Sending get-items request.
[DBG] (cl_request_process:506) get-items request sent, waiting for response.
[DBG] (cl_request_process:519) get-items response received, processing.
[DBG] (sr_get_item_next:1095) All items has been read for xpath '/kea-dhcp4-server:*//.'
[DBG] (srcfg_import_datastore:887) <LYD_DIFF_CHANGED> orig: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/subnet, new: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/subnet
[DBG] (cl_request_process:493) Sending set-item request.
[DBG] (cl_request_process:506) set-item request sent, waiting for response.
[DBG] (cl_request_process:519) set-item response received, processing.
[DBG] (srcfg_import_datastore:881) <LYD_DIFF_DELETED> node: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/pools/pool[start-address='10.0.0.50'][end-address='10.0.0.54']
[DBG] (cl_request_process:493) Sending delete-item request.
[DBG] (cl_request_process:506) delete-item request sent, waiting for response.
[DBG] (cl_request_process:519) delete-item response received, processing.
[DBG] (srcfg_import_datastore:895) <LYD_DIFF_CREATED> parent: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/pools, new node: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/pools/pool[start-address='192.168.51.50'][end-address='192.168.51.54']
[DBG] (cl_request_process:493) Sending set-item request.
[DBG] (cl_request_process:506) set-item request sent, waiting for response.
[DBG] (cl_request_process:519) set-item response received, processing.
[DBG] (srcfg_convert_lydiff_created:613) Skipping default node 'option-data-list'.
[DBG] (srcfg_import_datastore:899) <LYD_DIFF_MOVEDAFTER2> after: (null), this new node was inserted: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/pools/pool[start-address='192.168.51.50'][end-address='192.168.51.54']
[DBG] (cl_request_process:493) Sending move-item request.
[DBG] (cl_request_process:506) move-item request sent, waiting for response.
[DBG] (cl_request_process:519) move-item response received, processing.
[DBG] (cl_request_process:493) Sending commit request.
[DBG] (cl_request_process:506) commit request sent, waiting for response.
[DBG] (cl_request_process:519) commit response received, processing.
[ERR] (cl_request_process:543) Error by processing of the commit request (session id=82104949): Timeout has expired.
[ERR] (sr_commit:2078) Commit operation failed with 0 error(s).
[ERR] (srcfg_import_datastore:921) Error returned from sr_commit: Timeout has expired.
[ERR] (srcfg_import_datastore:924) (null) : Timeout has expired
Errors were encountered during importing. Cancelling the operation.
[DBG] (cl_request_process:493) Sending session-stop request.
[DBG] (cl_request_process:506) session-stop request sent, waiting for response.
[DBG] (cl_request_process:519) session-stop response received, processing.

Timeout? what happened with kea-netconf? It crashed:

terminate called after throwing an instance of 'sysrepo_exception'
  what():  Request contains unknown element
Aborted

Another important note, when I try to upload config to 'running' datastore after kea-netconf crashed:

wlodek@debian9-64-2:~/installed$ sudo sysrepocfg -l 4 -d startup -f xml -i /home/wlodek/installed/git/etc/kea/yang.xml kea-dhcp4-server
[DBG] (sr_log_stderr:141) Setting log level for stderr logs to 4.
[DBG] (sr_connect:380) Connecting to Sysrepo Engine.
[DBG] (cl_socket_connect:376) Connecting to socket=/tmp/sysrepod.sock
[INF] (sr_connect:435) Connected to daemon Sysrepo Engine at socket=/tmp/sysrepod.sock
[DBG] (cl_version_verify:431) Sending version-verify request.
[DBG] (cl_version_verify:442) version-verify request sent, waiting for response.
[DBG] (cl_version_verify:454) version-verify response received, processing.
[DBG] (cl_request_process:493) Sending session-start request.
[DBG] (cl_request_process:506) session-start request sent, waiting for response.
[DBG] (cl_request_process:519) session-start response received, processing.
[DBG] (srcfg_ly_init:189) Loading module schema: '/opt/sysrepo-0.7.5/build/repository/yang/kea-dhcp4-server@2018-09-14.yang'.
[DBG] (cl_request_process:493) Sending discard-changes request.
[DBG] (cl_request_process:506) discard-changes request sent, waiting for response.
[DBG] (cl_request_process:519) discard-changes response received, processing.
[DBG] (cl_request_process:493) Sending get-items request.
[DBG] (cl_request_process:506) get-items request sent, waiting for response.
[DBG] (cl_request_process:519) get-items response received, processing.
[DBG] (cl_request_process:493) Sending get-items request.
[DBG] (cl_request_process:506) get-items request sent, waiting for response.
[DBG] (cl_request_process:519) get-items response received, processing.
[DBG] (sr_get_item_next:1095) All items has been read for xpath '/kea-dhcp4-server:*//.'
[DBG] (srcfg_import_datastore:887) <LYD_DIFF_CHANGED> orig: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/subnet, new: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/subnet
[DBG] (cl_request_process:493) Sending set-item request.
[DBG] (cl_request_process:506) set-item request sent, waiting for response.
[DBG] (cl_request_process:519) set-item response received, processing.
[DBG] (srcfg_import_datastore:881) <LYD_DIFF_DELETED> node: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/pools/pool[start-address='10.0.0.50'][end-address='10.0.0.54']
[DBG] (cl_request_process:493) Sending delete-item request.
[DBG] (cl_request_process:506) delete-item request sent, waiting for response.
[DBG] (cl_request_process:519) delete-item response received, processing.
[DBG] (srcfg_import_datastore:895) <LYD_DIFF_CREATED> parent: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/pools, new node: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/pools/pool[start-address='192.168.51.50'][end-address='192.168.51.54']
[DBG] (cl_request_process:493) Sending set-item request.
[DBG] (cl_request_process:506) set-item request sent, waiting for response.
[DBG] (cl_request_process:519) set-item response received, processing.
[DBG] (srcfg_convert_lydiff_created:613) Skipping default node 'option-data-list'.
[DBG] (srcfg_import_datastore:899) <LYD_DIFF_MOVEDAFTER2> after: (null), this new node was inserted: /kea-dhcp4-server:config/subnet4/subnet4[id='1']/pools/pool[start-address='192.168.51.50'][end-address='192.168.51.54']
[DBG] (cl_request_process:493) Sending move-item request.
[DBG] (cl_request_process:506) move-item request sent, waiting for response.
[DBG] (cl_request_process:519) move-item response received, processing.
[DBG] (cl_request_process:493) Sending commit request.
[DBG] (cl_request_process:506) commit request sent, waiting for response.
[DBG] (cl_request_process:519) commit response received, processing.
The new configuration was successfully applied.
[DBG] (cl_request_process:493) Sending session-stop request.
[DBG] (cl_request_process:506) session-stop request sent, waiting for response.
[DBG] (cl_request_process:519) session-stop response received, processing.

it worked The new configuration was successfully applied but kea-netconf is dead so actually nothing happens.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information