Kea issueshttps://gitlab.isc.org/isc-projects/kea/-/issues2018-11-07T06:16:52Zhttps://gitlab.isc.org/isc-projects/kea/-/issues/153Netconf agent development2018-11-07T06:16:52ZFrancis DupontNetconf agent developmentHome for MRs about netconf.Home for MRs about netconf.Kea1.5-beta1Francis DupontFrancis Duponthttps://gitlab.isc.org/isc-projects/kea/-/issues/65library for yang <-> json configuration element translators2018-11-05T09:44:05ZGhost Userlibrary for yang <-> json configuration element translatorsNew library with yang <-> json translators for representation of configuration elements.
A priori one fro `src/lib/dhcpsrv/parsers` parse class.New library with yang <-> json translators for representation of configuration elements.
A priori one fro `src/lib/dhcpsrv/parsers` parse class.Kea1.5-beta1Francis DupontFrancis Duponthttps://gitlab.isc.org/isc-projects/kea/-/issues/156debian9 64 compilation error2021-08-31T21:20:50ZWlodzimierz Wenceldebian9 64 compilation error```
uname -a
Linux debian9-64-2 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u4 (2018-08-21) x86_64 GNU/Linux
autoreconf -if && ./configure --enable-shell --with-pgsql --with-mysql --prefix=/home/wlodek/installed/git/
```
result:
```
Packa...```
uname -a
Linux debian9-64-2 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u4 (2018-08-21) x86_64 GNU/Linux
autoreconf -if && ./configure --enable-shell --with-pgsql --with-mysql --prefix=/home/wlodek/installed/git/
```
result:
```
Package:
Name: kea
Version: 1.4.0-git
Extended version: 1.4.0-git (git 3fc0d776ff2262cae0013de1ce4c776c54c32eba)
OS Family: Linux
Hooks directory: /home/wlodek/installed/git/lib/hooks
Premium hooks: yes
Included Hooks: forensic_log flex_id host_cmds subnet_cmds radius host_cache
C++ Compiler:
CXX: g++
CXX_VERSION: g++ (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
CXX_STANDARD: 201402
DEFS: -DHAVE_CONFIG_H
CPPFLAGS: -DOS_LINUX -DBOOST_ASIO_HEADER_ONLY
CXXFLAGS: -g -O2
LDFLAGS: -lpthread
KEA_CXXFLAGS: -Wall -Wextra -Wnon-virtual-dtor -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -pthread -Wno-missing-field-initializers -fPIC
Python:
PYTHON: /usr/bin/python3
PYTHON_VERSION: 3.5
Boost:
BOOST_VERSION: 1.62
BOOST_INCLUDES:
BOOST_LIBS: -lboost_system
OpenSSL:
CRYPTO_VERSION: OpenSSL 1.1.0f 25 May 2017
CRYPTO_CFLAGS:
CRYPTO_INCLUDES:
CRYPTO_LDFLAGS:
CRYPTO_LIBS: -lcrypto
Botan: no
Log4cplus:
LOG4CPLUS_VERSION: 1.1.2
LOG4CPLUS_INCLUDES: -I/usr/include
LOG4CPLUS_LIBS: -L/usr/lib -L/usr/lib64 -llog4cplus
Flex/bison:
FLEX: flex
BISON: bison -y
MySQL:
MYSQL_VERSION: 10.1.26
MYSQL_CPPFLAGS: -I/usr/include/mysql
MYSQL_LIBS: -L/usr/lib/x86_64-linux-gnu -lmariadbclient -lpthread -lz -lm -ldl
PostgreSQL:
PGSQL_VERSION: PostgreSQL 9.6.10
PGSQL_CPPFLAGS: -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include/tcl8.6 -I/usr/include/postgresql -I/usr/include/postgresql/9.6/server
PGSQL_LIBS: -L/usr/lib/x86_64-linux-gnu -lpq
```
compilation error:
```
make[6]: Entering directory '/home/wlodek/kea/src/bin/agent'
CXXLD kea-ctrl-agent
../../../src/lib/process/.libs/libkea-process.so: undefined reference to `isc::db::DatabaseConnection::toElementDbAccessString(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
collect2: error: ld returned 1 exit status
Makefile:666: recipe for target 'kea-ctrl-agent' failed
make[6]: *** [kea-ctrl-agent] Error 1
```
It looks like this change:
```
diff --git a/src/bin/agent/Makefile.am b/src/bin/agent/Makefile.am
index 31a2798d8..75059f357 100644
--- a/src/bin/agent/Makefile.am
+++ b/src/bin/agent/Makefile.am
@@ -83,6 +83,7 @@ kea_ctrl_agent_LDADD += $(top_builddir)/src/lib/log/libkea-log.la
kea_ctrl_agent_LDADD += $(top_builddir)/src/lib/util/threads/libkea-threads.la
kea_ctrl_agent_LDADD += $(top_builddir)/src/lib/util/libkea-util.la
kea_ctrl_agent_LDADD += $(top_builddir)/src/lib/exceptions/libkea-exceptions.la
+kea_ctrl_agent_LDADD += $(top_builddir)/src/lib/database/libkea-database.la
kea_ctrl_agent_LDADD += $(LOG4CPLUS_LIBS) $(CRYPTO_LIBS) $(BOOST_LIBS)
kea_ctrl_agent_LDFLAGS = $(AM_LDFLAGS) $(CRYPTO_LDFLAGS)
```
fix issueKea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/134Small bugs from the new Xcode 10.0 compiler2018-09-27T12:49:19ZFrancis DupontSmall bugs from the new Xcode 10.0 compilerI updated yesterday the macOS Xcode to 10.0 and it finds now a few new warnings and I am using --with--werror...I updated yesterday the macOS Xcode to 10.0 and it finds now a few new warnings and I am using --with--werror...Kea1.5-beta1Francis DupontFrancis Duponthttps://gitlab.isc.org/isc-projects/kea/-/issues/129global reservation mode is DHCPv4 only.2019-09-02T07:12:00ZFrancis Dupontglobal reservation mode is DHCPv4 only.Looks like something forgotten as there is a global entry in reservations.Looks like something forgotten as there is a global entry in reservations.Kea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/259libyang unit test using doc examples is broken by new authoritative flag2018-11-08T11:56:07ZFrancis Dupontlibyang unit test using doc examples is broken by new authoritative flagKea1.5-beta1Francis DupontFrancis Duponthttps://gitlab.isc.org/isc-projects/kea/-/issues/256kea return error when receive 'get-config' command after being reconfigured w...2018-11-08T03:33:57ZWlodzimierz Wencelkea return error when receive 'get-config' command after being reconfigured with kea-netconfYang configuration:
```
<?xml version="1.0" ?>
<config xmlns="urn:ietf:params:xml:ns:yang:kea-dhcp4-server">
<interfaces-config>
<interfaces>enp0s9</interfaces>
</interfaces-config>
<client-classes...Yang configuration:
```
<?xml version="1.0" ?>
<config xmlns="urn:ietf:params:xml:ns:yang:kea-dhcp4-server">
<interfaces-config>
<interfaces>enp0s9</interfaces>
</interfaces-config>
<client-classes/>
<expired-leases-processing/>
<option-data-list>
<option-data>
<code>2</code>
<space>dhcp4</space>
<data>50</data>
<name>time-offset</name>
</option-data>
</option-data-list>
<control-socket>
<socket-name>/home/wlodek/installed/git/var/kea/control_socket</socket-name>
<socket-type>unix</socket-type>
</control-socket>
<lease-database>
<database-type>memfile</database-type>
<name>/home/wlodek/installed/git/var/kea/kea-leases4.csv</name>
</lease-database>
<subnet4>
<subnet4>
<id>1</id>
<subnet>192.168.51.0/24</subnet>
<valid-lifetime>4000</valid-lifetime>
<renew-timer>1000</renew-timer>
<rebind-timer>2000</rebind-timer>
<interface>enp0s9</interface>
<option-data-list/>
<relay>
<ip-addresses>192.12.11.4</ip-addresses>
</relay>
<pools>
<pool>
<start-address>192.168.51.50</start-address>
<end-address>192.168.51.50</end-address>
<option-data-list/>
</pool>
</pools>
</subnet4>
</subnet4>
</config>
```
Kea uses this configuration and assign address 192.168.51.50, but when I send 'get-config' to kea4 it sends back:
```
{
"result": 1,
"text": "Error during command processing: unknown DB access parameter: max-reconnect-tries=0"
}
```
and logs:
```
2018-11-07 08:02:00.045 INFO [kea-dhcp4.commands/5027] COMMAND_RECEIVED Received command 'config-get'
2018-11-07 08:02:00.045 WARN [kea-dhcp4.commands/5027] COMMAND_PROCESS_ERROR2 Error while processing command: unknown DB access parameter: max-reconnect-tries=0
```
When I checked logs from reconfiguration it looks like not everything went 100% correct:
```
2018-11-07 07:58:21.431 INFO [kea-dhcp4.dhcpsrv/5027] DHCPSRV_CFGMGR_NEW_SUBNET4 a new subnet has been added to configuration: 192.168.51.0/24 with params: t1=1000, t2=2000, valid-lifetime=4000
2018-11-07 07:58:21.431 INFO [kea-dhcp4.dhcp4/5027] DHCP4_CONFIG_COMPLETE DHCPv4 server has completed configuration: added IPv4 subnets: 1; DDNS: disabled
2018-11-07 07:58:21.431 INFO [kea-dhcp4.dhcpsrv/5027] DHCPSRV_MEMFILE_DB opening memory file lease database: lfc-interval=3600 max-reconnect-tries=0 name=/home/wlodek/installed/git/var/kea/kea-leases4.csv persist=true readonly=false reconnect-wait-time=0 tcp-keepalive=0 tcp-nodelay=true type=memfile universe=4
2018-11-07 07:58:21.431 INFO [kea-dhcp4.dhcpsrv/5027] DHCPSRV_MEMFILE_LEASE_FILE_LOAD loading leases from file /home/wlodek/installed/git/var/kea/kea-leases4.csv
```
Why values readonly=false reconnect-wait-time=0 tcp-keepalive=0 tcp-nodelay=true max-reconnect-tries=0 are added to memfile configuration?Kea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/254empty class data in class-update command causes assert in boost2018-11-08T11:17:14ZMichal Nowikowskiempty class data in class-update command causes assert in boostcommand:
```
{"command": "class-update", "arguments": {"client-classes": [{}]})
```
casuses such assert in boost:
```
kea-dhcp4: /usr/include/boost/smart_ptr/shared_ptr.hpp:734: typename boost::detail::sp_member_access<T>::type boost::sh...command:
```
{"command": "class-update", "arguments": {"client-classes": [{}]})
```
casuses such assert in boost:
```
kea-dhcp4: /usr/include/boost/smart_ptr/shared_ptr.hpp:734: typename boost::detail::sp_member_access<T>::type boost::shared_ptr<T>::operator->() const [with T = const isc::data::Element; typename boost::detail::
sp_member_access<T>::type = const isc::data::Element*]: Assertion `px != 0' failed.
```Kea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/116get rid of interface-id for DHCPv42018-09-19T12:51:11ZFrancis Dupontget rid of interface-id for DHCPv4Defined in the DHCPv4 syntax but only for subnets (not shared networks as in DHCPv6) and unused.Defined in the DHCPv4 syntax but only for subnets (not shared networks as in DHCPv6) and unused.Kea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/63kea-dhcp4 server needs to support configurable client hostname sanitization2018-09-06T12:34:16ZGhost Userkea-dhcp4 server needs to support configurable client hostname sanitizationThe Infoblox SOW requires the ability to sanitize client host name values, sent by DHCPv4 clients, prior to using them to form the FQDN for DNS updates. Specifically, there should be two configuration parameters:
1. A regular expressio...The Infoblox SOW requires the ability to sanitize client host name values, sent by DHCPv4 clients, prior to using them to form the FQDN for DNS updates. Specifically, there should be two configuration parameters:
1. A regular expression (e.g. [A-za-z0-9]-) which describes the allowed characters
2. A replacement for characters that are disallowed (e.g. _ by -)
Does not apply to v6, or to v4 FQDN option values, as per RFC they cannot contain non-compliant characters.Kea1.5-beta1Thomas MarkwalderThomas Markwalderhttps://gitlab.isc.org/isc-projects/kea/-/issues/49During traffic spikes that exceed Kea's throughput capacity, handle backlog m...2018-11-07T05:52:39ZGhost UserDuring traffic spikes that exceed Kea's throughput capacity, handle backlog more effectivelyThe current Kea implementation processes the inbound socket buffer as a simple queue - first in, first out. When the server is under pressure and not handling client packets as fast as they are arriving, a backlog will build up.
If t...The current Kea implementation processes the inbound socket buffer as a simple queue - first in, first out. When the server is under pressure and not handling client packets as fast as they are arriving, a backlog will build up.
If the situation continues for long enough, the client packets that the server is handling will have already timed-out on the client side, so it is pointless to spend time processing them and moreover wasting time on these old packets prevents the server from handling newer packets until they too have timed out. Effectively, it stops responding to active clients because it never gets through the backlog fast enough to reach the most recent inbounds.
Even though the initial spike in traffic may have subsided, the degraded performance can mean that clients change their behaviour, adding retries to the backlog and/or reverting back to initial discovery - thus increasing the backlog of packets to be processed and making recovery unlikely without restarting the server to clear things down.
We need to handle this situation better so that even when swamped, Kea servers are able to process a proportion of recently-received client packets, instead of none of them because it's 'stuck' with the oldest ones instead.
Suggestions being mooted so far suggest either an independent socket reading thread (or process) to manage the inbound traffic and to pull it off the sockets/interfaces on which the Kea server is listening. This will prevent the UDP buffers from overflowing as well as allowing the socket reader to apply better logic to:
- discarding the oldest client packets in favour of the most recently received
- managing the 'waiting' buffers appropriately to the throughput capacity of the server
Maximum per-server throughput will be highly dependent on both configuration and the choice of back-end (e.g database, or memfile, and if database, how and where etc..) - so it would be good to have the I/O handler be tunable too - not discarding too soon for a fast server and so on.
There's no clear operational mitigation strategy for this, other than ensuring sufficient headroom when provisioning so that there are no peaks in client traffic that can overwhelm the server(s) maximum capacity.
(Notably, increasing inbound UDP buffers is likely to make the situation worse rather than better.)Kea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/42Congestion handling2018-11-08T14:15:12ZGhost UserCongestion handlingI propose two ways to control the receive queue to avoid big backlogs which can happen when servicing is too slow (and the fact clients retransmit of course does not help at all).
First is the POSIX `setsockopt(SO_RCVBUF)` which sets th...I propose two ways to control the receive queue to avoid big backlogs which can happen when servicing is too slow (and the fact clients retransmit of course does not help at all).
First is the POSIX `setsockopt(SO_RCVBUF)` which sets the maximum size in bytes of the socket receive queue. When the queue is full (i.e. an incoming packet is bigger than the maximum minus the current size) new packets are dropped instead added at the end of the queue.
This allows to avoid big backlog but as it drops new packets it is not the best/only solution.
Second idea is to use `ioctl(FIONREAD)` which returns the current size in bytes of the receive queue (very efficient system call BTW). I propose to use it in two ways:
- when it returns a large value (threshold to determine) packets should be simply popped and dropped.
- after servicing a packet it is more efficient to look at if there is another one than to come back to select (a real performance pig). Of course only a limited (another parameter to determine) number of packets should be serviced because the select loop includes other services.
About the last part of the second idea I refer to the AFTR code where I implemented this.
Note there is a big theoretical and practical background on the way to manage queue in high load / congestion situations, e.g RED (Random Early Detection). A good subject for a student...
The earlier issue that covered initial discussion and some experiments is #49. Adding the number for easier reference.Kea1.5-beta1Thomas MarkwalderThomas Markwalderhttps://gitlab.isc.org/isc-projects/kea/-/issues/194libkea-process unnecessarily links with MySQL, PgSQL and CQL2018-11-01T10:35:39ZMarcin Siodelskilibkea-process unnecessarily links with MySQL, PgSQL and CQLPreviously, libkea-process depended on libkea-dhcpsrv. Therefore, it had to conditionally link with MySQL and co. After we reversed the dependency, so as the libkea-dhcpsrv now depends on libkea-process, linking with MySQL is no longer n...Previously, libkea-process depended on libkea-dhcpsrv. Therefore, it had to conditionally link with MySQL and co. After we reversed the dependency, so as the libkea-dhcpsrv now depends on libkea-process, linking with MySQL is no longer needed.Kea1.5-beta1Marcin SiodelskiMarcin Siodelskihttps://gitlab.isc.org/isc-projects/kea/-/issues/148lib process servers without arguments2018-10-19T22:25:15ZFrancis Dupontlib process servers without argumentsAll servers that use CPL architecture (D2, CA, netconf) are unable to print out usage information when run without any parameters. Instead, they print out cryptic error message as shown below:
```$ kea-ctrl-agent
2018-10-01 22:28:51.285...All servers that use CPL architecture (D2, CA, netconf) are unable to print out usage information when run without any parameters. Instead, they print out cryptic error message as shown below:
```$ kea-ctrl-agent
2018-10-01 22:28:51.285 FATAL [kea-ctrl-agent.dctl/23508] DCTL_PID_FILE_ERROR
Control-agent could not create a PID file: Daemon::makePIDFileName config file name is not set
Service failed: Launch failed: Daemon::makePIDFileName config file name is not set'''Kea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/127Distribute yang models2018-10-19T09:56:06ZFrancis DupontDistribute yang modelsKea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/92Move common classes pertaining to databases from libkea-dhcpsrv to their own ...2018-12-06T23:10:33ZMarcin SiodelskiMove common classes pertaining to databases from libkea-dhcpsrv to their own librariesConfiguration Backend structure described here: https://gitlab.isc.org/isc-projects/kea/wikis/designs/configuration-in-db-design#configuration-backend-structure calls for moving some of the existing classes from libkea-dhcpsrv to new lib...Configuration Backend structure described here: https://gitlab.isc.org/isc-projects/kea/wikis/designs/configuration-in-db-design#configuration-backend-structure calls for moving some of the existing classes from libkea-dhcpsrv to new libraries. For example: DatabaseConnection, DbLogger should go to libkea-database. This ticket covers such refactoring.Kea1.5-beta1Thomas MarkwalderThomas Markwalderhttps://gitlab.isc.org/isc-projects/kea/-/issues/89CB: Create MySQL schema and upgrade scripts2018-09-18T06:09:26ZMarcin SiodelskiCB: Create MySQL schema and upgrade scriptsThis ticket covers updating MySQL schema to facilitate Config Backend feature for 1.5.0 release:
https://gitlab.isc.org/isc-projects/kea/wikis/designs/configuration-in-db-design#mysql
I adds new tables, scripts and upgrade scripts to M...This ticket covers updating MySQL schema to facilitate Config Backend feature for 1.5.0 release:
https://gitlab.isc.org/isc-projects/kea/wikis/designs/configuration-in-db-design#mysql
I adds new tables, scripts and upgrade scripts to MySQL.Kea1.5-beta1Marcin SiodelskiMarcin Siodelskihttps://gitlab.isc.org/isc-projects/kea/-/issues/83Update HA design with the ability to hold state2018-08-27T17:14:26ZGhost UserUpdate HA design with the ability to hold stateWe're going to implement a hold in a waiting state capability in HA.
As a first step, we need to to update the [requirements](../wikis/HARequriements) and the [design](../wikis/HADesign). This ticket covers this update.We're going to implement a hold in a waiting state capability in HA.
As a first step, we need to to update the [requirements](../wikis/HARequriements) and the [design](../wikis/HADesign). This ticket covers this update.Kea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/78Extend HA hooks library to synchronize leases by chunks, i.e. multiple fetche...2018-11-05T19:25:30ZGhost UserExtend HA hooks library to synchronize leases by chunks, i.e. multiple fetches of leasesOne of the major use cases for the lease_cmds hooks library is to provide a way to synchronize leases between HA enabled servers. Currently the HA hooks library will fetch the entire lease database which requires the lease_cmds hooks lib...One of the major use cases for the lease_cmds hooks library is to provide a way to synchronize leases between HA enabled servers. Currently the HA hooks library will fetch the entire lease database which requires the lease_cmds hooks library to create a JSON structure of the whole lease database. This eats the CPU and memory. In case of large number of leases in the database it may freeze the server for a long period of time.
In order to mitigate this issue the lease_cmds hooks librart must support fetching limited number of leases, e.g. 1000, 2000 leases etc. The controlling client should be able to specify last fetched leases with the limit and the server should return leases with addresses beyond this last fetched address. That way, the entire lease database may be returned in chunks with client specifying the start of the next chunk.
This ticket is about extending the HA hooks library to utilize this mechanism implemented with #5651 in the lease_cmds.Kea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/67Need a mechanism to output binary expressions as hexadecimal strings2019-09-02T07:12:00ZGhost UserNeed a mechanism to output binary expressions as hexadecimal stringsA prospective user attempted to use the following expression for a value of a Radius attribute:
{
"name": "Password"
"expr": "pkt4.mac"
}
The value produced by pkt4.mac, is a series of binary bytes, which cannot be
expressed in a...A prospective user attempted to use the following expression for a value of a Radius attribute:
{
"name": "Password"
"expr": "pkt4.mac"
}
The value produced by pkt4.mac, is a series of binary bytes, which cannot be
expressed in a Radius authorization file. The radius server actually errors out stating the values do not match. We need a way to convert an expression result to a string literal such as:
"0xXXXXX" or "xx:xx:xx"
We could do it as perhaps a function:
hexString(pkt4.mac)
This would be useful beyond this particular case involving Radius.
As an aside using the same expression for User Name or Connection Id work because we convert it to a string, under the covers.Kea1.5-beta1Stephen MorrisStephen Morris