Kea issueshttps://gitlab.isc.org/isc-projects/kea/-/issues2024-02-16T13:28:24Zhttps://gitlab.isc.org/isc-projects/kea/-/issues/1Update NETCONF requirements and design2024-02-16T13:28:24ZTomek MrugalskiUpdate NETCONF requirements and designAs the first step, we need to expand [the requirements](../wikis/designs/netconf-requirements) and [the design](../wikis/designs/netconf-design) pages. These are living documents, so they probably will never be truly done.
Nevertheless,...As the first step, we need to expand [the requirements](../wikis/designs/netconf-requirements) and [the design](../wikis/designs/netconf-design) pages. These are living documents, so they probably will never be truly done.
Nevertheless, the goal of this ticket is to have them sufficiently complete, so the code implementation could start and there's realistic expectation that other parties interested in the code (QA team, external users) could have realistic expectation what they would get.Kea1.5-beta1Francis DupontFrancis Duponthttps://gitlab.isc.org/isc-projects/kea/-/issues/3215Changing the value of a key in a YANG list element creates a new node rather ...2024-01-25T14:52:44ZAndrei Pavelandrei@isc.orgChanging the value of a key in a YANG list element creates a new node rather than replacing itThe title may be over-generalizing, but I suspect not.
I experienced this issue with option-data for kea-dhcp4.
The issue manifests in master and 2.4.1 with sysrepo v2, but also in 2.2.1 with sysrepo v1.
The issue may become more prev...The title may be over-generalizing, but I suspect not.
I experienced this issue with option-data for kea-dhcp4.
The issue manifests in master and 2.4.1 with sysrepo v2, but also in 2.2.1 with sysrepo v1.
The issue may become more prevalent if issue 3198 gets merged as it was written at the time this issue was created. It makes `data` a key and that makes it more likely to have entries with only code, space and data (all keys), which is also why this issue became obvious there.
Replication:
1. Start kea-dhcp4 with a control-socket configured.
2. Start kea-netconf with the same control-socket configured under the `"dhcp4"` server.
3. Do a `sysrepocfg --edit=kea-dhcp4-config.xml` where `kea-dhcp4-config.xml` has this content:
```xml
<config xmlns="urn:ietf:params:xml:ns:yang:kea-dhcp4-server">
<control-socket>
<socket-name>/tmp/kea-dhcp4-ctrl.sock</socket-name>
<socket-type>unix</socket-type>
</control-socket>
<option-data>
<code>100</code>
<space>dhcp4</space>
<data>1234</data>
</option-data>
</config>
```
4. Run the command again with `<code>100</code>` changed to `<code>101</code>`.
Only one option was expected as specified in the XML, but there are two options as indicated by the kea-netconf logs:
```
DEBUG [kea-netconf.netconf] NETCONF_UPDATE_CONFIG updating configuration with dhcp4 server: {
"Dhcp4": {
"control-socket": {
"socket-name": "/tmp/kea-dhcp4-ctrl.sock",
"socket-type": "unix"
},
"option-data": [
{
"code": 100,
"data": "1234",
"space": "dhcp4"
},
{
"code": 101,
"data": "1234",
"space": "dhcp4"
}
]
}
}
INFO [kea-netconf.netconf] NETCONF_UPDATE_CONFIG_COMPLETED completed updating configuration for dhcp4 server
```
and by the response to a subsequent `config-get` command which contains:
```
"option-data": [
{
"always-send": false,
"code": 100,
"csv-format": true,
"data": "1234",
"name": "tcode",
"space": "dhcp4"
},
{
"always-send": false,
"code": 101,
"csv-format": true,
"data": "1234",
"name": "pcode",
"space": "dhcp4"
}
],
```https://gitlab.isc.org/isc-projects/kea/-/issues/2311Any plans to update netconf support package from libyang1 to libyang2?2024-01-22T08:33:33ZShinAny plans to update netconf support package from libyang1 to libyang2?
As indicated by
"https://kea.readthedocs.io/en/latest/arm/integrations.html"
"To get its NETCONF capabilities, Kea uses libyang v1.0.240 and Sysrepo v1.4.140. Use packages if they are provided by the system."
Currently ISC kea support l...
As indicated by
"https://kea.readthedocs.io/en/latest/arm/integrations.html"
"To get its NETCONF capabilities, Kea uses libyang v1.0.240 and Sysrepo v1.4.140. Use packages if they are provided by the system."
Currently ISC kea support libyang version 1.
Do we have any plans to update it to libyang2?
Currently latest libyang version is Version 2.0.112
https://github.com/CESNET/libyang/releases/tag/v2.0.112kea2.3.2Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/3199Minor Netconf documentation issue2024-01-11T15:07:21ZDarren AnkneyMinor Netconf documentation issueAs of the 2.4.1 version of the documentation, the ARM shows `output-options` in the example `kea-netconf` configurations found under https://kea.readthedocs.io/en/kea-2.4.1/arm/integrations.html#yang-netconf The `kea-netconf` daemon refu...As of the 2.4.1 version of the documentation, the ARM shows `output-options` in the example `kea-netconf` configurations found under https://kea.readthedocs.io/en/kea-2.4.1/arm/integrations.html#yang-netconf The `kea-netconf` daemon refuses to start until these are changed to `output_options` when using Kea 2.4.1 matching the version of the ARM.https://gitlab.isc.org/isc-projects/kea/-/issues/3014Unable to complete make when tryin to install NETCONF2023-09-21T13:18:55ZSandeep GagalapallyUnable to complete make when tryin to install NETCONF---
name: Bug report
about: make is stuck when installing NETCONF
---
**Describe the bug**
Unable to complete make when tryin to install NETCONF
**To Reproduce**
Steps to reproduce the behavior:
Followed this documentation : https://...---
name: Bug report
about: make is stuck when installing NETCONF
---
**Describe the bug**
Unable to complete make when tryin to install NETCONF
**To Reproduce**
Steps to reproduce the behavior:
Followed this documentation : https://kea.readthedocs.io/en/kea-2.4.0/arm/integrations.html#installing-netconf
```
git clone gitlab.isc.org/isc-projects/kea.git
cd kea
autoreconf -f -i
./configure --with-libyang --with-libyang-cpp --with-sysrepo --with-sysrepo-cpp
make - Stuck at this step !!
```
Stuck here
make[5]: Entering directory '/home/ubuntu/kea/src/lib/dhcp'
CXX libkea_dhcp___la-iface_mgr.lo
**Expected behavior**
Complete NETCONF Installation with no issues.
**Environment:**
- Kea version: 2.0.2
- OS: [Ubuntu 22.04.3 LTS amd64]
- Which features were compiled in (in particular which backends)
- If/which hooks where loaded in
**Additional Information**
Error: make[5]: Entering directory '/home/ubuntu/kea/src/lib/dhcp'
CXX libkea_dhcp___la-iface_mgr.lo
**Contacting you**
Please contact via email at sgagalap@gmail.comhttps://gitlab.isc.org/isc-projects/kea/-/issues/2558fixes for YANG unit tests2023-07-17T13:58:24ZAndrei Pavelandrei@isc.orgfixes for YANG unit testsThe recent expansion of distcheck scope revealed some unit test problems:
* Alpines:
```sh
../../../../../../../src/share/yang/modules/utils/check-revisions.sh: line 27: yanglint: not found
revision mismatch on keatest-module@2018-11-20...The recent expansion of distcheck scope revealed some unit test problems:
* Alpines:
```sh
../../../../../../../src/share/yang/modules/utils/check-revisions.sh: line 27: yanglint: not found
revision mismatch on keatest-module@2018-11-20.yang got
FAIL: check-revisions.sh
```
* RPM distros:
```sh
yanglint: error while loading shared libraries: libyang.so.1: cannot open shared object file: No such file or directory
revision mismatch on kea-types@2019-08-12.yang got
FAIL: check-revisions.sh
```
https://jenkins.aws.isc.org/job/kea-dev/job/distcheck/869/
These also happen on ut-extended, but because we run those with `make check -k ...`, they don't stop the job and the errors are not put in the report and not detected.kea2.3.1Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/2400improve performance of NETCONF operations by retrieving items in bulk2023-07-17T13:58:24ZAndrei Pavelandrei@isc.orgimprove performance of NETCONF operations by retrieving items in bulkData often needs to be retrieved from a sysrepo datastore. Such an action is done, among other opportunities, every time a change is brought to the configuration. kea-netconf currently calls `Session::get_item()` for all leaves and `Sess...Data often needs to be retrieved from a sysrepo datastore. Such an action is done, among other opportunities, every time a change is brought to the configuration. kea-netconf currently calls `Session::get_item()` for all leaves and `Session::get_items()` for all leaf lists. The repeated calls to these functions hinder performance as shown by the call graph below[0] which is collected after a minor change is done through NETCONF to a configuration with 114 subnets. This action takes 10 seconds on my laptop which might seem reasonable, but it can take significantly more on lesser provisioned systems. The performance also seems to worsen quadratically. See [RT#20676](https://support.isc.org/Ticket/Display.html?id=20676) for a distribution of measurements over configuration sizes.
The performance could be improved by doing a single run to the sysrepo datastore and retrieving the entire configuration data in one go. Retrieving the entire configuration seems to be a requirement because we validate it before comitting any minor change. If this had not been a requirement, we could only retrieve the changed node and apply it to the already committed configuration.
As such, the bulk retrieval could be done via `sr_get_items()` / `Session::get_items()` on the top-level node. The efficiency of this approach is also mentioned in Sysrepo code:
https://github.com/sysrepo/sysrepo/blob/v1.4.140/src/sysrepo.h#L749-L751
```
* @see Use ::sr_get_items for retrieving larger chunks
* of data from the datastore. Since it retrieves the data from datastore in
* larger chunks, it can work much more efficiently than multiple ::sr_get_item calls.
```
Or, in the case that it only works on leaf lists, then `sr_get_data()` / `Session::get_data()` would be the alternative:
https://github.com/sysrepo/sysrepo/blob/v1.4.140/src/sysrepo.h#L805-L806
```
* Top-level trees are always returned so if an inner node is selected, all of its descendants
* and its direct parents (lists also with keys) are returned.
```
[0] Click and zoom. See how most of the time is spent in `sr_get_item` and `lyb_parse_subtree`.
![sysrepo-call-graph](/uploads/d3495dee5000dfb108cc0afe23bb362c/sysrepo-call-graph.png)kea2.3.2Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/2601Refactor NETCONF code after migrating to sysrepo22023-07-17T13:58:23ZAndrei Pavelandrei@isc.orgRefactor NETCONF code after migrating to sysrepo2Issue 2311 migrated the code to sysrepo2, but did a minimum amount of changes required to do so. Because of that, there are some parts of the code that are not completely adjusted to the new dependencies. Here are some of the proposed ch...Issue 2311 migrated the code to sysrepo2, but did a minimum amount of changes required to do so. Because of that, there are some parts of the code that are not completely adjusted to the new dependencies. Here are some of the proposed changes that would make the code more cohesive or that would just refactor the code for improved readability:
* The basic translators might not be able to translate all data types. The migration was done in a best-effort approach to apease the unit tests. But there is a noticeable lack of tests for unions and leafrefs for example. `keatest-module` has unions, but only one variant of it is tested. There is [this discussion in the MR from 2311](https://gitlab.isc.org/isc-projects/kea/-/merge_requests/1811#note_323844) that details one test case.
* Consider removing the `LeafBaseType` argument from `TranslatorBasic::value()`. The information available in the given `ElementPtr` might be enough to do the translation.
* Rename `s_val` to `data_node` where appropriate. The data type was changed from `S_Value` to `DataNode`, but the variable name was left the same to minimize changes. They are not equivalent. The former represents the value of a leaf or leaf-list member. The latter can represent an entire tree of data.
* Have references with meaningful names to `service_pair.first` and `service_pair.second` in `src/bin/netconf`. It's confusing what is one and what is the other. The first one is the module name, and the other is the kea-netconf configuration, I think.
* Rename `SysrepoError` to `NetconfError`. Errors can come from both libyang and sysrepo, so let's be inclusive.
* Consider changing `ConstElementPtr`s to `ElementPtr`s in `src/lib/yang` and `src/bin/netconf` and, with this, remove the need for `const_pointer_cast`. I like to squabble about the undefined behavior that these present, and if we're not removing them from everywhere, I would like to have it removed at least in the code that I am responsible for.
* Consider renaming methods in `TranslatorBasic`. I think their names can be more representative of what they do, especially after the migration.
* Solve doxygen warnings about functions with the same name in `src/lib/yang`.
* Use `checkAndGetLeaf` and `checkAndSetLeaf` where possible in translators.
* `decode64` and `encode64` are free-standing functions in `translator.cc`, but they can just as well be private methods inside `TranslatorBasic`.
* Consider doing something about `NETCONF_SUBSCRIBE_NOTIFICATIONS_FAILED`. It's an error message that shows up for the Kea modules because they don't have any notification nodes. kea-netconf continues regardless. We could at least turn it into a warning message.
* Solve the few TODOs in `src/lib/yang`.
* Replace `EXPECT_NO_THROW` with `EXPECT_NO_THROW_LOG` in `src/lib/yang` and `src/bin/netconf` for more insight when tests fail.
* Replace `EXPECT_THROW` with `EXPECT_THROW_MSG` in `src/lib/yang` and `src/bin/netconf` for stricter checking.kea2.3.3Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/2617Wdeprecated-declarations warnings on std::iterator2023-07-17T13:58:23ZAndrei Pavelandrei@isc.orgWdeprecated-declarations warnings on std::iteratorStarted getting these warnings after the migration to sysrepo v2. These should be all of them:
```
encode/base_n.cc:170:33: warning: ‘template<class _Category, class _Tp, class _Distance, class _Pointer, class _Reference> struct std::i...Started getting these warnings after the migration to sysrepo v2. These should be all of them:
```
encode/base_n.cc:170:33: warning: ‘template<class _Category, class _Tp, class _Distance, class _Pointer, class _Reference> struct std::iterator’ is deprecated [-Wdeprecated-declarations]
170 | class DecodeNormalizer : public iterator<input_iterator_tag, char> {
| ^~~~~~~~
encode/base_n.cc:115:33: warning: ‘template<class _Category, class _Tp, class _Distance, class _Pointer, class _Reference> struct std::iterator’ is deprecated [-Wdeprecated-declarations]
115 | class EncodeNormalizer : public iterator<input_iterator_tag, uint8_t> {
| ^~~~~~~~
../../../src/lib/dns/message.h:91:37: warning: ‘template<class _Category, class _Tp, class _Distance, class _Pointer, class _Reference> struct std::iterator’ is deprecated [-Wdeprecated-declarations]
91 | class SectionIterator : public std::iterator<std::input_iterator_tag, T> {
| ^~~~~~~~
../../../src/lib/dns/rrset_collection_base.h:156:27: warning: ‘template<class _Category, class _Tp, class _Distance, class _Pointer, class _Reference> struct std::iterator’ is deprecated [-Wdeprecated-declarations]
156 | class Iterator : std::iterator<std::forward_iterator_tag,
| ^~~~~~~~
```
[The paper](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0174r2.html#2.1) that deprecated it seems to have done so because of the bad practice of having to derive classes in the `std` library.
Proposing to fix them. After minimal research, it would seem that removing the inheritance and having a few using declarations inside the formerly-derived class is sufficient e.g.:
```
using iterator_category = std::input_iterator_tag;
using value_type = T;
```kea2.3.4Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/2832Add missing YANG configuration knobs for the Kea 2.4.0 release2023-07-17T13:58:21ZAndrei Pavelandrei@isc.orgAdd missing YANG configuration knobs for the Kea 2.4.0 releaseThese knobs are missing from the Kea YANG modules:
- `allocator`
- `ddns-ttl-percent`
- `exclude-first-last-24`
- `ignore-dhcp-server-identifier`
- `offer-lifetime`
- `pd-allocator`
- `read-timeout`
- `tcp-user-timeout`
- `write-timeout`These knobs are missing from the Kea YANG modules:
- `allocator`
- `ddns-ttl-percent`
- `exclude-first-last-24`
- `ignore-dhcp-server-identifier`
- `offer-lifetime`
- `pd-allocator`
- `read-timeout`
- `tcp-user-timeout`
- `write-timeout`kea2.3.8Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/kea/-/issues/2667sysrepo-cpp and libyang-cpp flags are not propagated to distcheck (after migr...2023-05-16T12:55:56ZRazvan Becheriusysrepo-cpp and libyang-cpp flags are not propagated to distcheck (after migrating code to sysrepo2)current-stable-2.4https://gitlab.isc.org/isc-projects/kea/-/issues/7Implement libyang library2022-10-27T12:44:25ZTomek MrugalskiImplement libyang libraryThis task covers adding a libyang library. It has at least provide:
- makefile changes to build a new lib
- unit-tests
- translation utilities for netconf primitives (int, string, bool, etc) to JSON and vice versa
- a base class for tra...This task covers adding a libyang library. It has at least provide:
- makefile changes to build a new lib
- unit-tests
- translation utilities for netconf primitives (int, string, bool, etc) to JSON and vice versa
- a base class for translator
- a base class for watcher (a piece of code that exposes a callback that can be called when certain part of netconf tree changes)Kea1.5-beta1Tomek MrugalskiTomek Mrugalskihttps://gitlab.isc.org/isc-projects/kea/-/issues/6Simplify CPL framework to be more suitable for kea-netconf2022-10-27T12:44:25ZTomek MrugalskiSimplify CPL framework to be more suitable for kea-netconfThe CPL framework that was initially designed and developed for D2 and was later used for CA is considered superior to what DHCPv4 and DHCPv6 use. However it has a number of disadvantages that should be mitigated:
- way too many classes...The CPL framework that was initially designed and developed for D2 and was later used for CA is considered superior to what DHCPv4 and DHCPv6 use. However it has a number of disadvantages that should be mitigated:
- way too many classes needed (agent, classes derived derived from controller, process, DCfgContextBase, DCfgMgrBase)
- lack of common class to store logging information (Daemon from libdhcpsrv is used for this, resulting in the need to include libdhcpsrv library everywhere)Kea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/5Configuration parser for NETCONF2022-10-27T12:44:25ZTomek MrugalskiConfiguration parser for NETCONFThis task covers writing configuration parser for kea-netconf. This configuration will cover things like:
- which model(s) to subscribe to
- which translators to load
- where send the JSON commands (stdout, unix socket, http socket)
- l...This task covers writing configuration parser for kea-netconf. This configuration will cover things like:
- which model(s) to subscribe to
- which translators to load
- where send the JSON commands (stdout, unix socket, http socket)
- loggingKea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/4Write basic documentation for netconf2022-10-27T12:44:25ZTomek MrugalskiWrite basic documentation for netconfTo enable early engagement of various people, it is essential to write a simple documentation. It's ok it to be very basic at the beginning, but it must at the very least cover:
- how to compile Kea with netconf
- some basic tutorial ho...To enable early engagement of various people, it is essential to write a simple documentation. It's ok it to be very basic at the beginning, but it must at the very least cover:
- how to compile Kea with netconf
- some basic tutorial how to load YANG models to sysrepo
- how to load configuration to those YANG modelsKea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/3Implement kea-netconf skeleton2022-10-27T12:44:25ZTomek MrugalskiImplement kea-netconf skeletonThe first programmatic step will be to implement a skeleton kea-netconf daemon. At the very minimum the code has to:
1. be able to detect sysrepo location and link with it.
2. have a standalone daemon (kea-netconf) that can accept an in...The first programmatic step will be to implement a skeleton kea-netconf daemon. At the very minimum the code has to:
1. be able to detect sysrepo location and link with it.
2. have a standalone daemon (kea-netconf) that can accept an input config file (and do nothing with it yet)
3. the daemon must be able to log something. For the time being, printing out a version number and startup message is enough.
4. there has to be a boilerplate manual page for it.
5. there have to be unit-tests, at least one shell test and at least one gtest based test.Kea1.5-beta1Tomek MrugalskiTomek Mrugalskihttps://gitlab.isc.org/isc-projects/kea/-/issues/2Document NETCONF translator commands2022-10-27T12:44:25ZTomek MrugalskiDocument NETCONF translator commandsOnce #1 is done, the next step will be to write down which specific NETCONF structures (from kea-dhcpv4-server and ietf-dhcpv6-server models) would be converted to which JSON commands that Kea could understand.
This is an essential docu...Once #1 is done, the next step will be to write down which specific NETCONF structures (from kea-dhcpv4-server and ietf-dhcpv6-server models) would be converted to which JSON commands that Kea could understand.
This is an essential document for anyone who is interested in integrating kea-netconf with any external tools or systems.Kea1.5-beta1https://gitlab.isc.org/isc-projects/kea/-/issues/2143./configure --with-libyang passes but should fail when libyang is not installed2022-10-21T20:13:42ZAndrei Pavelandrei@isc.org./configure --with-libyang passes but should fail when libyang is not installedkea2.3.2https://gitlab.isc.org/isc-projects/kea/-/issues/945YANG model and library code update2022-09-21T13:24:57ZFrancis DupontYANG model and library code update#199 and just merged #35 modified the server syntax so an update is required. IMHO this should be addressed as late as possible (but not too late so just before a release, and according to previous history before the beta).
Assume it wi...#199 and just merged #35 modified the server syntax so an update is required. IMHO this should be addressed as late as possible (but not too late so just before a release, and according to previous history before the beta).
Assume it will be for a 1.7.x so creating in 1.7 backlog.outstandinghttps://gitlab.isc.org/isc-projects/kea/-/issues/2136Update yang model with valid-lifetime for classes2021-11-18T20:55:14ZPeter DaviesUpdate yang model with valid-lifetime for classesUpdate yang model with valid-lifetime for classes :
The latest kea-dhcp4 yang model for netconf does not include “valid-lifetime” for classes, which was introduced in Kea 1.9.11.
RT [#19492](https://support.isc.org/Ticket/Display.htm...Update yang model with valid-lifetime for classes :
The latest kea-dhcp4 yang model for netconf does not include “valid-lifetime” for classes, which was introduced in Kea 1.9.11.
RT [#19492](https://support.isc.org/Ticket/Display.html?id=19492)kea2.1.1Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.org