stork issueshttps://gitlab.isc.org/isc-projects/stork/-/issues2021-03-02T18:22:51Zhttps://gitlab.isc.org/isc-projects/stork/-/issues/407automate combining ChangeLog into release notes2021-03-02T18:22:51ZMichal Nowikowskiautomate combining ChangeLog into release notesoutstandinghttps://gitlab.isc.org/isc-projects/stork/-/issues/597CI: solve TODO about automating changelog update in release notes2022-10-05T14:46:17ZAndrei Pavelandrei@isc.orgCI: solve TODO about automating changelog update in release notes```
# TODO:
# - automate pasting ChangeLog.md to release notes
``````
# TODO:
# - automate pasting ChangeLog.md to release notes
```outstandinghttps://gitlab.isc.org/isc-projects/stork/-/issues/587IPv6 support in Stork system test framework2023-07-27T12:26:19ZSlawek FigielIPv6 support in Stork system test frameworkThe system tests framework incorrectly handles IPv6 addresses. It causes a system test building to fail on some configurations.
We need to rewrite our scripts to support this IP schema.
We noticed also that some system tests were execut...The system tests framework incorrectly handles IPv6 addresses. It causes a system test building to fail on some configurations.
We need to rewrite our scripts to support this IP schema.
We noticed also that some system tests were executed, but produce incorrect results when IPv6 was used. We need to prepare some unit and system tests to cover IPv6-based configurations.
```
distro_agent = 'ubuntu/18.04', distro_server = 'centos/8'
@pytest.mark.parametrize("distro_agent, distro_server", SUPPORTED_DISTROS)
def test_pkg_upgrade_server_token(distro_agent, distro_server):
"""Check if Stork agent and server can be upgraded from latest release
to localy built packages."""
server = containers.StorkServerContainer(alias=distro_server)
agent = containers.StorkAgentContainer(alias=distro_agent)
# install the latest version of stork from cloudsmith
server.setup_bg('cloudsmith')
while server.mgmt_ip is None:
time.sleep(0.1)
agent.setup_bg('cloudsmith', server.mgmt_ip)
server.setup_wait()
agent.setup_wait()
# login
r = server.api_post('/sessions', json=dict(useremail='admin', userpassword='admin'), expected_status=200) # TODO: POST should return 201
assert r.json()['login'] == 'admin'
# install local packages
banner('UPGRADING STORK SERVER')
server.prepare_stork_server()
# get server token from server
for i in range(100):
try:
r = server.api_get('/machines-server-token')
break
except:
if i == 99:
raise
time.sleep(1)
data = r.json()
server_token = data['token']
# install kea on the agent machine
agent.install_kea()
# install local packages using server token based way
banner('UPGRADING STORK AGENT')
server_url = 'http://%s:8080' % server.mgmt_ip
> agent.run('curl -o stork-install-agent.sh %s/stork-install-agent.sh' % server_url)
agent = <containers.StorkAgentContainer object at 0x7fdd88d29970>
data = {'token': 'o85LV4YpOCqapgfaiZ4feeoUJL4fiw8v'}
distro_agent = 'ubuntu/18.04'
distro_server = 'centos/8'
i = 0
r = <Response [200]>
server = <containers.StorkServerContainer object at 0x7fdd88bfa040>
server_token = 'o85LV4YpOCqapgfaiZ4feeoUJL4fiw8v'
server_url = 'http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080'
tests.py:211:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <containers.StorkAgentContainer object at 0x7fdd88d29970>, cmd = 'curl -o stork-install-agent.sh http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080/stork-install-agent.sh'
env = {'LANG': 'en_US.UTF-8', 'LANGUAGE': 'en_US:UTF-8', 'LC_ALL': 'en_US.UTF-8'}, ignore_error = False, attempts = 1, sleep_time_after_attempt = None
def run(self, cmd, env=None, ignore_error=False, attempts=1, sleep_time_after_attempt=None):
cmd2 = shlex.split(cmd)
if env is None:
env = {}
env['LANG'] = "en_US.UTF-8"
env['LANGUAGE'] = "en_US:UTF-8"
env['LC_ALL'] = "en_US.UTF-8"
for attempt in range(attempts):
result = self.cntr.execute(cmd2, env)
out = 'run: %s\n' % cmd
out += result[1]
self._trace_logs(out, 'out')
self._trace_logs(result[2], 'err')
if result[0] == 0:
break
elif attempt < attempts - 1:
print('command failed, retry, attempt %d/%d' % (attempt, attempts))
if sleep_time_after_attempt:
time.sleep(sleep_time_after_attempt)
if result[0] != 0 and not ignore_error:
> raise Exception('problem with cmd: %s' % cmd)
E Exception: problem with cmd: curl -o stork-install-agent.sh http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080/stork-install-agent.sh
attempt = 0
attempts = 1
cmd = 'curl -o stork-install-agent.sh http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080/stork-install-agent.sh'
cmd2 = ['curl', '-o', 'stork-install-agent.sh', 'http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080/stork-install-agent.sh']
env = {'LANG': 'en_US.UTF-8', 'LANGUAGE': 'en_US:UTF-8', 'LC_ALL': 'en_US.UTF-8'}
ignore_error = False
out = 'run: curl -o stork-install-agent.sh http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080/stork-install-agent.sh\n'
result = InstanceExecuteResult(exit_code=3, stdout='', stderr="curl: (3) Port number ended with ':'\n")
self = <containers.StorkAgentContainer object at 0x7fdd88d29970>
sleep_time_after_attempt = None
containers.py:228: Exception
```outstandinghttps://gitlab.isc.org/isc-projects/stork/-/issues/804Process: Enable SAST (and other tools) in CI2024-01-10T16:09:24ZTomek MrugalskiProcess: Enable SAST (and other tools) in CIGitlab provides SAST (Static Application Security Testing) and Secret Detection that can be enabled in CI.
We should enable both of them.Gitlab provides SAST (Static Application Security Testing) and Secret Detection that can be enabled in CI.
We should enable both of them.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/811Limit maintenance downtime during deploying the demo2022-10-25T13:33:56ZSlawek FigielLimit maintenance downtime during deploying the demoCurrently, to deploy a demo, we first shut down an old version, and next, we build and run a new one. The build takes ~1 hour. The demo is not available at this time. We can refactor our solution to build the demo in the first order and ...Currently, to deploy a demo, we first shut down an old version, and next, we build and run a new one. The build takes ~1 hour. The demo is not available at this time. We can refactor our solution to build the demo in the first order and restart it next. It should significantly limit maintenance downtime.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/841Remove the annoying danger complaint about the commit message line length of ...2023-09-26T09:55:20ZMarcin SiodelskiRemove the annoying danger complaint about the commit message line length of 50 characters and dot presenceWe have discussed in #827 that we may potentially try to reconfigure danger to not require us to fit the commit message within the 50 characters line length. I am not in favor of allowing very very long commit messages but 50 characters ...We have discussed in #827 that we may potentially try to reconfigure danger to not require us to fit the commit message within the 50 characters line length. I am not in favor of allowing very very long commit messages but 50 characters seems short to me. If we can reconfigure it to 70-80 characters, fine. I'd be also fine with disabling this particular check. @slawek says it is possible.
We can also consider ignoring the warning message, so that the reviewer doesn't have to bother checking if the commit messages are fine by danger.outstandingMarcin SiodelskiMarcin Siodelskihttps://gitlab.isc.org/isc-projects/stork/-/issues/853Add APK packages to "deploy_pkgs" CI job2022-11-15T16:06:33ZMarcin GodzinaAdd APK packages to "deploy_pkgs" CI jobAdd APK packages to "deploy_pkgs" CI jobAdd APK packages to "deploy_pkgs" CI jobbackloghttps://gitlab.isc.org/isc-projects/stork/-/issues/862Pre-release CI pipelines2022-12-13T12:58:56ZSlawek FigielPre-release CI pipelinesI'm introducing in #817 the possibility of running system tests with different Kea and Bind9 versions.
Our standard system test pipeline now uses Kea 2.0 and Bind9 9.18. But we can prepare additional CI tasks/pipelines to test other conf...I'm introducing in #817 the possibility of running system tests with different Kea and Bind9 versions.
Our standard system test pipeline now uses Kea 2.0 and Bind9 9.18. But we can prepare additional CI tasks/pipelines to test other configurations.
Unfortunately, the system tests pipeline executes quite long ~15 minutes. It is inconvenient to run it many times for every pushed commit. But we can run the additional CI pipelines only for pre-releases merge requests, i.e., merge requests that pump the Stork version. They are usually merged after code freeze but a day before sanity checks. We should have enough time to check the bugs found.
I think the pre-release pipelines may also contain the installation and de-installation tests.
There should be a possibility to run the pipelines manually on demand.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/1157Process: Investigate and enable Pyre/mypy and Pysa, if useful2023-09-21T10:16:32ZTomek MrugalskiProcess: Investigate and enable Pyre/mypy and Pysa, if usefulAs @manu reported in his [security audit](https://gitlab.isc.org/isc-private/kea/-/wikis/Kea-Security-Review-02-2023#7-github-security-features-pyrepysa-related-to-stork), we might take a look at Pyre or mypy (type checker) and Pysa (sta...As @manu reported in his [security audit](https://gitlab.isc.org/isc-private/kea/-/wikis/Kea-Security-Review-02-2023#7-github-security-features-pyrepysa-related-to-stork), we might take a look at Pyre or mypy (type checker) and Pysa (static analyzer) that could improve our code quality.
- [ ] Pyre/mypy
- [ ] Pysa
If the tools are unsuitable, please document the reasons here and close the ticket.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/1161Linter and formatter for Ruby2023-09-22T10:07:43ZSlawek FigielLinter and formatter for RubyOut Rakefiles (written in Ruby) are no longer trivial. We should add the Ruby linter and formatter to force the consistent and proper coding style for these files.Out Rakefiles (written in Ruby) are no longer trivial. We should add the Ruby linter and formatter to force the consistent and proper coding style for these files.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/1181Hooks in system tests2024-02-06T14:41:53ZSlawek FigielHooks in system testsThere should be a possibility to write the system tests for hooks and use hook in demo.There should be a possibility to write the system tests for hooks and use hook in demo.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/1224e2e tests: remove protractor, decide next steps2024-02-02T12:56:44ZTomek Mrugalskie2e tests: remove protractor, decide next stepsWhen Stork's angular project was created a long time ago (in 2020), it added the default end-2-end tests with protractor. AFAICT this was never used. The protractor tool has reached [EOL in Aug 2023](https://www.protractortest.org/#/). W...When Stork's angular project was created a long time ago (in 2020), it added the default end-2-end tests with protractor. AFAICT this was never used. The protractor tool has reached [EOL in Aug 2023](https://www.protractortest.org/#/). We should clean up this outdated mess and...
- [x] get rid of Protractor (and likely all of the default files in `webui/e2e`)
- [ ] decide if we want to develop end 2 end tests
- [ ] if the answer is yes, evaluate [alternatives](https://www.browserstack.com/guide/protractor-alternatives)
- [ ] add a few e2e tests using new testing framework
As for now, the five leading alternatives seem to be Nightwatch.js (11.5k), Cypress (45.2k), Playwright (56.6k), TestCafe (9.7k), WebDriverIO (8.4k). The number in brackets show number of stars on github, which is some indicator of popularity. Other aspects to consider: some frameworks use Selenium (which @wlodek had bad experiences with). Playwright is developed by Microsoft, which might be a factor. The link above contains example test cases.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/1291QA: Consider secret scanning2024-02-07T11:24:58ZTomek MrugalskiQA: Consider secret scanningTo prevent token and other secrets leaking, we might consider deploying secret scanning tools. Here's two that we might consider:
- [trufflehog](https://github.com/trufflesecurity/trufflehog) -13.2k stars on github
- [gitleaks](https://...To prevent token and other secrets leaking, we might consider deploying secret scanning tools. Here's two that we might consider:
- [trufflehog](https://github.com/trufflesecurity/trufflehog) -13.2k stars on github
- [gitleaks](https://gitleaks.io/) - 14.6k stars on github
There are plenty others, too.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/1312Follow-up from the 1.15 release2024-03-28T13:47:02ZAndrei Pavelandrei@isc.orgFollow-up from the 1.15 releaseIssues with the release that need to be fixed:
- The `upload_to_repo` job was not able to trigger any gitlab runner after the migration in issue 1170. The job has too many tags. Removing `stork-repo` or `aws` would solve the impasse.
- ...Issues with the release that need to be fixed:
- The `upload_to_repo` job was not able to trigger any gitlab runner after the migration in issue 1170. The job has too many tags. Removing `stork-repo` or `aws` would solve the impasse.
- The release notes are currently folded at 73 characters. This includes folding changelog entryes. There seems to be a gentleman's agreement that changelog entries should be capped at 80 characters. This makes the changelog in the release notes less readable than it could be. Look at the [1.14 release notes](https://downloads.isc.org/isc/stork/1.14.0/Stork-1.14.0-ReleaseNotes.txt) for example. This has been manually fixed for 1.15, but extending the folding to 80 would be good. A lint step in CI to make sure this rule is respected for changelog entries wouldn't hurt either.
-1.16Piotrek ZadrogaPiotrek Zadrogahttps://gitlab.isc.org/isc-projects/stork/-/issues/1348Remove stork-repo tag from .gitlab-ci.yaml2024-03-27T16:57:15ZSlawek FigielRemove stork-repo tag from .gitlab-ci.yamlThe `upload_to_repo` CI task is not working as described [on MM QA channel](https://mattermost.isc.org/isc/pl/eoq3yfamnbfz7n6tfaewdmc6mh).
We got this error:
> This job is stuck because of one of the following problems. There are no ac...The `upload_to_repo` CI task is not working as described [on MM QA channel](https://mattermost.isc.org/isc/pl/eoq3yfamnbfz7n6tfaewdmc6mh).
We got this error:
> This job is stuck because of one of the following problems. There are no active runners online, no runners for the protected branch, or no
runners that match all of the job's tags:
Its tags defined in the .gitlab-ci.yml file are:
- linux
- aws
- runner-manager
- amd64
- stork-repo
I checked the Stork repository settings and it seems there is no runner that matches the requirements.
The `aws` tag is missing in all runners that have the stork-repo tag.
### Proposed solution:
* remove `stork-repo` tag from `upload_to_repo` job in .gitlab-ci.yml.
* remove `-4` (which enforces IPv4 connection) from `ssh-keyscan -4 repo.isc.org >> ~/.ssh/known_hosts`