ISC Open Source Projects issueshttps://gitlab.isc.org/groups/isc-projects/-/issues2023-09-01T08:25:04Zhttps://gitlab.isc.org/isc-projects/bind9/-/issues/1181nsupdate: soa lookup should be using tcp when "-v" is given2023-09-01T08:25:04ZGhost Usernsupdate: soa lookup should be using tcp when "-v" is given### Description
`nsupdate` offers the switch `-v` to use tcp for update requests. But before sending that update request `nsupdate` gathers the SOA using a udp connection, and there's currently no way I know of to change that.
As `nsupd...### Description
`nsupdate` offers the switch `-v` to use tcp for update requests. But before sending that update request `nsupdate` gathers the SOA using a udp connection, and there's currently no way I know of to change that.
As `nsupdate` is already able to use a different port both for querying the SOA and for sending update requests, it'd be a nice feature to have `nsupdate` honor the `-v` option for the SOA query as well.
Use case: I'm running a dns-over-tls setup using bind together with stunnel. The tunnel's endpoint must be tcp and `nsupdate` is not able to get the SOA over that tunnel.
### Request
Please make `nsupdate` honor `-v` for SOA queries as well.
### Links / referencesSeptember 2023 (9.16.44, 9.16.44-S1, 9.18.19, 9.18.19-S1, 9.19.17)Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/keama/-/issues/424.5.0 post release version bump2023-09-20T13:47:07ZMarcin Godzina4.5.0 post release version bumpBump versions after 4.5.0 release to 4.5.1-git
This ends code freeze.Bump versions after 4.5.0 release to 4.5.1-git
This ends code freeze.4.5.0Marcin GodzinaMarcin Godzinahttps://gitlab.isc.org/isc-projects/keama/-/issues/364.5.0 - Release checklist2023-09-21T06:09:08ZMarcin Godzina4.5.0 - Release checklist# Keama Release Checklist
1. Gitlabab pipeline results:
1. [x] Check Gitlab [build](https://gitlab.isc.org/isc-projects/keama/-/pipelines) job for failures
1. [x] Check Gitlab [pytest](https://gitlab.isc.org/isc-projects/keama/...# Keama Release Checklist
1. Gitlabab pipeline results:
1. [x] Check Gitlab [build](https://gitlab.isc.org/isc-projects/keama/-/pipelines) job for failures
1. [x] Check Gitlab [pytest](https://gitlab.isc.org/isc-projects/keama/-/pipelines) job for failures
1. Tarball preparation:
1. [x] If this is the release of the final version, please check the sanity check ticket of the previous release and make sure all comments are addressed
1. [x] bump up version in configure.ac
1. [x] bump up version in `tests/test_keama.py:test_version()` (isc-projects/keama#40)
1. [ ] Note release in changelog
1. [x] check the date in LICENSE
1. [x] Check README file (including installation details)
1. [x] update copyrights in all touched files using a simple script in [qa-dhcp](https://gitlab.isc.org/isc-private/qa-dhcp/-/tree/master/dhcp/scripts).
1. [x] Do `autoreconf -i` on a modern system, test the code compiles and push your changes.
1. Build tarball, packages and docker image
1. [x] Go to [keama-tarball](https://jenkins.aws.isc.org/view/isc-dhcp-dev/job/dhcp-dev/job/keama-tarball/) > Build with Parameters, in field `keamaBranch` put in release branch and run the job, this will build release tarball and save it as an artifact of the job
1. [x] Go to [keama-pkg](https://jenkins.aws.isc.org/view/isc-dhcp-dev/job/dhcp-dev/job/keama-pkg/) > Build with Parameters, in field `keamaBranch` put in release branch and run the job, this will build release packages and save it as an artifact of the job (`baseBranch` should be 'master' if not using custom qa-dhcp branch)
1. [x] Go to [keama-docker](https://jenkins.aws.isc.org/view/isc-dhcp-dev/job/dhcp-dev/job/keama-pkg/) > Build with Parameters, in field `keamaBranch` put in release branch and run the job, this will build release tarball and save it as an artifact of the job (`keamadockerBranch` should be 'master' if not using custom keama-docker branch)
1. [x] Go to [keama-release-notes](https://jenkins.aws.isc.org/view/isc-dhcp-dev/job/dhcp-dev/job/keama-release-notes/) > Build with Parameters, in field `version` put in release version and run the job, this will build release notes in txt format and save it as an artifact of the job.
1. [x] before tarball will be deemed as ready to release it will be `release candidate`. Each consecutive respin will have it's own name starting from `-rc1`
1. [x] open a ticket in keama repo called `release X.Y.Z-rcX sanity checks` and put there location of release tarball, packages, and docker image
1. [x] wait for team input about new tarball, if respin is needed go back to `Build tarball` point also increasing release candidate number
1. [x] if tarball is accepted create a tag of this version on a last commit in release branch and kea-docker repo
1. [x] move tarball and release notes to non release candidate location (e.g. moving to /data/shared/sweng/keama/releases/4.5.0)
1. [x] make sure that new release directory allow group write e.g. `chmod 665 /data/shared/sweng/dhcp/releases/4.3.2b1`
1. [x] upload packages to cloudsmith
1. [x] upload docker image to cloudsmith
1. [x] open tickets to address issues mentioned in sanity checks IF those were not already fixed and close sanity check ticket
1. Signing and notification
1. [x] Sign the tarball and put signature files with the tarball
1. [x] notify support about readiness of release, at this point QA and dev team work is done
1. Post release
1. [x] mark jenkins jobs to save forever.
1. [x] Bump version in `configure.ac`, `tests/test_keama.py:test_version()` (isc-projects/keama#40)and do `autoreconf -i`
1. [x] push to repo and unfreeze the code
1. Releasing tarball
- [x] ***(Support)*** Wait for clearance from Security Officer to proceed with the public release (if applicable).
- [x] ***(Support)*** Confirm that the tarballs are signed correctly.
- [x] ***(Support)*** Place tarballs in public location on FTP site.
- [x] ***(Support)*** Publish links to downloads on ISC website.
- [x] ***(Support)*** Write release email to *dhcp-announce*.
- [x] ***(Support)*** Write email to *dhcp-users* (if a major release).
- [x] ***(Support)*** Update tickets in case of waiting for support customers.
- [x] ***(Marketing)*** Announce on social media.
- [ ] ***(Marketing)*** Write blog article (if a major release).4.5.0https://gitlab.isc.org/isc-projects/keama/-/issues/35Come up with a release procedure2023-09-16T19:02:20ZTomek MrugalskiCome up with a release procedureAs the source code is based on isc-dhcp, the natural choice would be to reuse `make dist`. However, there are couple things that are not covered:
- web interface (the whole `web/` dir)
- python tests
- much of the new documentation
- muc...As the source code is based on isc-dhcp, the natural choice would be to reuse `make dist`. However, there are couple things that are not covered:
- web interface (the whole `web/` dir)
- python tests
- much of the new documentation
- much of the code was removed
Also, this might be a good time to remove remaining legacy code (the server in particular). Initially, the server was left over as possibly an assistance for testing migration. But right now I think it's more of a nuisance. Users might incorrectly perceive this as some kind of extended life of `isc-dhcpd`...4.5.02023-09-20https://gitlab.isc.org/isc-projects/keama/-/issues/34Force enable CSRF in flask WTForms2023-09-14T14:23:48ZPiotrek ZadrogaForce enable CSRF in flask WTFormsLooking at `flask-wtf` docs (https://flask-wtf.readthedocs.io/en/1.1.x/csrf/):
> Any view using FlaskForm to process the request is already getting CSRF protection.
you could have an impression that using `FlaskForm` implies that CSRF ...Looking at `flask-wtf` docs (https://flask-wtf.readthedocs.io/en/1.1.x/csrf/):
> Any view using FlaskForm to process the request is already getting CSRF protection.
you could have an impression that using `FlaskForm` implies that CSRF is used.
In fact, we do send CSRF token in our form requests, but currently the token is not being checked on back end side.
CSRF must be enabled globally.4.5.0Piotrek ZadrogaPiotrek Zadrogahttps://gitlab.isc.org/isc-projects/keama/-/issues/33Do not generate the secret key every time Flask starts2023-09-14T14:23:48ZPiotrek ZadrogaDo not generate the secret key every time Flask startsCurrently we generate random `SECRET_KEY` every time `flask` starts. This has some negative side effects e.g. this breaks session cookies.
If we want to use `CSRF` protection for sending forms (we really should do this!) it will result ...Currently we generate random `SECRET_KEY` every time `flask` starts. This has some negative side effects e.g. this breaks session cookies.
If we want to use `CSRF` protection for sending forms (we really should do this!) it will result in random `CSRF` token fails (in this case `flask` usually responds with `400 Bad request`).
Details are described here - https://stackoverflow.com/questions/27287391/why-not-generate-the-secret-key-every-time-flask-starts
It is better practice to have in code some dummy and constant `SECRET_KEY` which makes `CSRF` work ok in DEV and in tests.
We can add a comment in code/docs that whenever one wants to deploy `keama web`, `SECRET_KEY` should be generated by e.g. UUID generator.4.5.0Piotrek ZadrogaPiotrek Zadrogahttps://gitlab.isc.org/isc-projects/keama/-/issues/31Change keama web interface to be inline with isc.org style2023-09-14T14:23:48ZPiotrek ZadrogaChange keama web interface to be inline with isc.org styleKeama web interface, especially the one hosted as https://dhcp.isc.org/, could look more like official isc.org page.Keama web interface, especially the one hosted as https://dhcp.isc.org/, could look more like official isc.org page.4.5.0Piotrek ZadrogaPiotrek Zadrogahttps://gitlab.isc.org/isc-projects/keama/-/issues/30The changelog should be presented on the web interface2023-09-14T14:23:48ZTomek MrugalskiThe changelog should be presented on the web interfaceKeama is evolving. Users should be able to view what's the latest changes. This will be particularly useful once we start fixing bugs reported by users.Keama is evolving. Users should be able to view what's the latest changes. This will be particularly useful once we start fixing bugs reported by users.4.5.0Piotrek ZadrogaPiotrek Zadrogahttps://gitlab.isc.org/isc-projects/keama/-/issues/29Improve feedback keama web is showing to the user2023-09-14T14:23:48ZPiotrek ZadrogaImprove feedback keama web is showing to the userAs it was mentioned in https://gitlab.isc.org/isc-projects/keama/-/merge_requests/16#note_382168 , feedback to the user could be displayed in a more dynamic manner i.e. without a need to reload whole page (http get request).
Flask flash...As it was mentioned in https://gitlab.isc.org/isc-projects/keama/-/merge_requests/16#note_382168 , feedback to the user could be displayed in a more dynamic manner i.e. without a need to reload whole page (http get request).
Flask flashing could be used for that purpose.4.5.0Piotrek ZadrogaPiotrek Zadrogahttps://gitlab.isc.org/isc-projects/keama/-/issues/26Tweak migrated.html template for migration fail cases2023-09-14T14:23:48ZPiotrek ZadrogaTweak migrated.html template for migration fail casesWhen using keama web interface for migration, when there is any error during migration then `migrated.html` template is rendered in the same way like in case of a success.
e.g. `download` and `share` links have `(empty)` file.
I think ...When using keama web interface for migration, when there is any error during migration then `migrated.html` template is rendered in the same way like in case of a success.
e.g. `download` and `share` links have `(empty)` file.
I think it should be rendered differently, have another feedback to the user, links should be hidden etc.4.5.0Piotrek ZadrogaPiotrek Zadrogahttps://gitlab.isc.org/isc-projects/keama/-/issues/25Improve share with ISC2023-09-14T14:23:48ZTomek MrugalskiImprove share with ISCSuggestions for web improvements:
- [x] save original file, not just the output (suggested by @manu)
- [x] feedback form (suggested by @tomek and @vicky)Suggestions for web improvements:
- [x] save original file, not just the output (suggested by @manu)
- [x] feedback form (suggested by @tomek and @vicky)4.5.0Tomek MrugalskiTomek Mrugalskihttps://gitlab.isc.org/isc-projects/keama/-/issues/23Fix or disable failing tests2023-09-14T14:23:48ZTomek MrugalskiFix or disable failing testsThere's a number of failing tests. Those should be investigated and either fixed or disabled.There's a number of failing tests. Those should be investigated and either fixed or disabled.4.5.0Tomek MrugalskiTomek Mrugalskihttps://gitlab.isc.org/isc-projects/keama/-/issues/21Keama web should timeout hanging processes2023-09-14T14:23:48ZTomek MrugalskiKeama web should timeout hanging processesThe web interface calls keama binary without any timeouts. If it goes into infinite loop, it will hang indefinitely.
There should be a timeout.The web interface calls keama binary without any timeouts. If it goes into infinite loop, it will hang indefinitely.
There should be a timeout.4.5.0Tomek MrugalskiTomek Mrugalskihttps://gitlab.isc.org/isc-projects/keama/-/issues/16feedback form for KeaMA web page2023-09-14T14:23:48ZVicky Riskvicky@isc.orgfeedback form for KeaMA web pageProposal:
We add this to the response template, and the form results will be sent to Tomek, and also stored in formspree.
------
```
<form id="fs-frm" name="simple-contact-form" accept-charset="utf-8" action="https://formspree.io/f/xay...Proposal:
We add this to the response template, and the form results will be sent to Tomek, and also stored in formspree.
------
```
<form id="fs-frm" name="simple-contact-form" accept-charset="utf-8" action="https://formspree.io/f/xayzgype"method="post">
<fieldset id="fs-frm-inputs">
<label for="email-address">Email Address for followup</label>
<input type="email" name="_replyto" id="email-address" placeholder="email@domain.tld" required="">
<label for="message">Feedback on the KeaMA configuration migration tool</label>
<textarea rows="5" name="message" id="message" placeholder="Did the tool work for you? Did it do what you expected? Do you plan to go ahead with a migration to Kea?" required=""></textarea>
<input type="hidden" name="_subject" id="email-subject" value="Contact Form Submission">
</fieldset>
<input type="submit" value="Submit feedback">
</form>4.5.0https://gitlab.isc.org/isc-projects/keama/-/issues/14Keama should be more verbose2023-09-14T14:23:48ZTomek MrugalskiKeama should be more verboseRight now keama is completely silent in positive case. It would be useful to write something to stdout when the conversion happens. Something like:
- "Reading input file xyz.conf"
- "Subnet 123.45.67.89/24 converted"
- "Hostname 11:22:3...Right now keama is completely silent in positive case. It would be useful to write something to stdout when the conversion happens. Something like:
- "Reading input file xyz.conf"
- "Subnet 123.45.67.89/24 converted"
- "Hostname 11:22:33:44:55:66 converted"
- "Written output to xyz.json"
This would be useful for users to know how the conversion went. When implementing the web interface, I've tried to capture stdout and stderr from keama, but couldn't make it print anything useful on stdout.4.5.0Tomek MrugalskiTomek Mrugalskihttps://gitlab.isc.org/isc-projects/keama/-/issues/13web interface for keama2023-09-14T14:23:48ZTomek Mrugalskiweb interface for keamaWe need a web interface for Keama. Let's do some experiments.We need a web interface for Keama. Let's do some experiments.4.5.0Wlodzimierz WencelWlodzimierz Wencelhttps://gitlab.isc.org/isc-projects/keama/-/issues/12keama should report its version2023-09-14T14:23:49ZTomek Mrugalskikeama should report its versionThis might seem a detail, but with more focus on users running `keama`, we want to have a way to understand which version the user had been running.This might seem a detail, but with more focus on users running `keama`, we want to have a way to understand which version the user had been running.4.5.0https://gitlab.isc.org/isc-projects/keama/-/issues/10Convert existing tests into more standard framework2023-09-14T14:23:48ZTomek MrugalskiConvert existing tests into more standard framework@tmark implemented tests for Keama that are shell based. These have their own custom way of running and it's unclear how to run them, no way to filter them, parse logs etc.
It would be much easier to integrate it with our QA processes i...@tmark implemented tests for Keama that are shell based. These have their own custom way of running and it's unclear how to run them, no way to filter them, parse logs etc.
It would be much easier to integrate it with our QA processes if there were implemented using some standard framework, such as gtest or pytest.4.5.0Marcin GodzinaMarcin Godzinahttps://gitlab.isc.org/isc-projects/keama/-/issues/8keama build fails on illumos because of incorrect headers2023-09-14T14:23:48ZGhost Userkeama build fails on illumos because of incorrect headerskeama build fails with
```
keama.c: In function 'main':
keama.c:127:33: error: 'errno' undeclared (first use in this function)
"input: %s", strerror(errno));
^~~~~
```
on OpenIndiana, because ...keama build fails with
```
keama.c: In function 'main':
keama.c:127:33: error: 'errno' undeclared (first use in this function)
"input: %s", strerror(errno));
^~~~~
```
on OpenIndiana, because sys/errno.h doesn't define errno.
The following patch helps.
[004-errno.patch](/uploads/5df1a6ea904a544f2444cac4d6895fa6/004-errno.patch)4.5.0https://gitlab.isc.org/isc-projects/keama/-/issues/6Keama doesn't build on Free BSD 12.12024-02-08T13:50:37ZPeter DaviesKeama doesn't build on Free BSD 12.1Keama doesn't build on Free BSD 12.1
./configure
make
...
cd keama
make
cc -DHAVE_CONFIG_H -I. -I../includes -g -O2 -Wall -Werror -fno-strict-aliasing -I../includes -I/tmp/dhcp-4.4.2/bind/include -MT keama.o -MD -MP -MF .deps/k...Keama doesn't build on Free BSD 12.1
./configure
make
...
cd keama
make
cc -DHAVE_CONFIG_H -I. -I../includes -g -O2 -Wall -Werror -fno-strict-aliasing -I../includes -I/tmp/dhcp-4.4.2/bind/include -MT keama.o -MD -MP -MF .deps/keama.Tpo -c -o keama.o keama.c
keama.c:75:19: error: use of undeclared identifier 'AF_INET'
local_family = AF_INET;
^
keama.c:77:19: error: use of undeclared identifier 'AF_INET6'
local_family = AF_INET6;
^
See [RT #17269](https://support.isc.org/Ticket/Display.html?id=17269)4.5.0Tomek MrugalskiTomek Mrugalski