Kea Release Cycle (draft)
This is a work in progress! Kea team is discussing the release processes. There may be some changes, but if you don't voice your concerns soon, this will be adopted and then any changes would be a bit more troublesome.
Historic release cycles (1.6.0 and earlier)
Kea 1.6.0 and earlier releases were going out roughly once every 6 months. Initially the date was roughly repetitive, but with increased number of customer requests, it was more and more difficult to deliver planned features on time and not neglect requests for support. This resulted with the releases not being that predictable.
Every time we did a release (1.4.0, 1.5.0, etc), it was always a feature release introducing new features. Versions ending with something else than 0 were very sporadic and done only when specific circumstances required us to release them.
For every feature release there was at least one beta release (done a month in advance). After beta was out, there was a memorandum on adding new features and only bugfixes and minor improvements could be committed between beta and final releases. In recent betas, we often slipped some features in. We very rarely received any feedback for betas, and even when we did, it was usually too late to do anything about it.
Every release was a major event. Getting closer to the release all developers and QA were stressed and overworked. Once the release was out, everyone was exhausted, was recovering and for some time the productivity was significantly down.
The 6 months cycle was too long. If the devs fixed a small bug, the user had to either wait up to 6 months for the next release or the devs had to go through patch creation process, then QA had to verify the patch applies correctly and often support team had to coordinate the patch delivery.
Our QA system, Jenkins, was testing code from git, but we were releasing tarballs. These two are similar, but not exactly the same things.
The last release done in that regime was be 1.6.0. It is also the first release designated maintenance under the new regime.
New Release Process (starting after 1.6.0)
There will be two baseline releases: stable and development. Stable will have the even minor version (e.g. 1.6.0, 1.6.1, 1.8.0 etc.). Development releases will have the odd minor version (e.g. 1.7.0, 1.7.1, some time in the future 1.9.0 etc.).
Development releases will go out every month, on the last Wednesday of the month. These will be 1.7.0, 1.7.1, 1.7.2 and so on.
Maintenance release will go out infrequently, on as needed basis. Typically we will release updated stable releases (such as 1.6.1) only when there's specific need, such as customer requiring a bugfix or we discover some serious bug that we feel is important enough for every user to go through the costly upgrade process.
Features will be planned for the next stable release (1.8) and then gradually implemented in monthly development releases (1.7.0, 1.7.1, etc.). Once all the features planned for 1.8 are implemented, we will release a stable 1.8.0. Depending on how the work progressed, this may be after 1.7.5 or perhaps 1.7.7. We do not make any assumptions on when that will happen. We will release it when it's ready.
There won't be any betas. In a sense all development releases are a series of betas.
Support policy: Our support policy will state we'll be supporting the last two major releases, e.g. we will support 1.4.0 and 1.5.0 until 1.6.0 is released. Once it is out, we will give people some time to upgrade and then will stop supporting 1.4.0.
Major benefits of the new approach
- We start testing what we're going to release (tarballs rather than git code).
- The release process becomes greatly simplified. We will be testing tarballs all the time, so the only actions needed for release would be: change version number, add a tag in git, get the tarball from jenkins.
- A release stops being a major event. If something doesn't make it, it's not a big deal anymore. The next chance is at most 31 days away.
- More evenly paced development (no high stress release phase followed by long recovery with decreased productivity)
- Fewer requests for patches - users and customers often can wait few weeks for the next dev release.
- Stop doing betas anymore. The feedback was disappointingly scarce and often too late.
- More aligned with what's BIND is doing.
Development process changes
Milestones in gitlab: There is one milestone that will cover all tickets we want to do in the whole 1.7 cycle (kea1.7). There will not be specific milestones for each month. Ok, there will be, but only for historical reasons. Once we release a development release, @tomek will create a new milestone for it, move all the resolved tickets to it and immediately close it. This way we can always go back and see what was actually done in specific dev release, but we won't be confused with distracting milestones. Gitlab does a good job with not showing completed milestones, unless you really want to see them.
For each development release we'll tag a specific commit and gitlab will be to create a new tag for it (see this form: https://gitlab.isc.org/isc-projects/kea/tags/new).
For stable releases, there will be a branch (e.g. v1_6_0). If we do v1_6_1, it will be branched from v1_6_0 with some fixes cherry-picked.
- There will be more releases. On the other hand, the burden of handling patches is expected to be decreased. Instead of going through patches, the answer will often be: please upgrade to x.y.z.
- Need to clarify our EOL policy: we will support last two major branches. Once 1.6.0 is out, we'll drop support for 1.4.x shortly afterwards.
- Need to come up with a good naming for the releases: development, preview or engineering drop are some candidates. I don't like unstable, because they'll not really be unstable (they'll go through the same QA process as every other release), more like some of the new features will be a work in progress.
- we will stop using tags for purpose other than releases.
Wlodek: A lot of my concerns went away when release is "tag on a branch" :) automating release process will be a challenge for example, before release I'm doing:
- check perflab for results
- check jenkins to make sure everything is ok
- check is there are no critical tickets left to do
- check known issues list if everything that was listed in previous version was fixed
- check lib versioning
- check hook version it will probably had to be done before each release.
Also we need there are some documentation checks:
- update supported platforms if needed
- update 3rd party software if needed
- check if we list and describe all binaries in sbin/bin, update if needed
- update if needed build requirements
- check Changelog for typos missing entries in core/premium
Automated steps at the moment:
- reorder *.mes files in core/premium
- check dates and versions in xml manuals
- check copy-right dates in edited files
Things I'd like to see are: