stork issueshttps://gitlab.isc.org/isc-projects/stork/-/issues2019-12-27T17:30:26Zhttps://gitlab.isc.org/isc-projects/stork/-/issues/29CI: pre-commit git hook2019-12-27T17:30:26ZTomek MrugalskiCI: pre-commit git hookWe need a hook that will put the [#issue number] in the commit message automatically. @fdupont may have such a script already.We need a hook that will put the [#issue number] in the commit message automatically. @fdupont may have such a script already.Stork-0.3Tomek MrugalskiTomek Mrugalskihttps://gitlab.isc.org/isc-projects/stork/-/issues/30CI: setup jenkins (or gitlab ci) env to run unit-tests automatically2019-10-24T08:12:46ZTomek MrugalskiCI: setup jenkins (or gitlab ci) env to run unit-tests automaticallyThese are still very early days. Let's see if there is anything we can deploy right now to kick off the CI process.These are still very early days. Let's see if there is anything we can deploy right now to kick off the CI process.Stork-0.1https://gitlab.isc.org/isc-projects/stork/-/issues/32CD (Continuous Delivery) - develop automated deployment to stork.lab.isc.org2020-06-22T08:29:23ZTomek MrugalskiCD (Continuous Delivery) - develop automated deployment to stork.lab.isc.orgDuring exec discussions about Stork, there was a request from Sales team to have something useful for a kind of demo system. We're still several months away from having anything useful enough to be users worthy, but we should start think...During exec discussions about Stork, there was a request from Sales team to have something useful for a kind of demo system. We're still several months away from having anything useful enough to be users worthy, but we should start thinking about our processes to deploy master (or perhaps selected builds) to stork.lab.isc.org.
The goal of this issue to came up with:
1. some rules proposal when to deploy updated code (every commit to master, every commit that passed jenkins tests, perhaps only manually?)
2. develop scripts that will make the deployment possible.Stork-0.3Michal NowikowskiMichal Nowikowskihttps://gitlab.isc.org/isc-projects/stork/-/issues/164Fix webui unit-tests2020-10-27T14:42:34ZTomek MrugalskiFix webui unit-testsI've tried to run webui unit-tests. Here's what I did:
```
rake build_ui
export CHROME_BIN=chromium
cd webui
npx ng test
```
An example error in the first comment.
We should:
1. investigate why the tests are failing. If they're non-s...I've tried to run webui unit-tests. Here's what I did:
```
rake build_ui
export CHROME_BIN=chromium
cd webui
npx ng test
```
An example error in the first comment.
We should:
1. investigate why the tests are failing. If they're non-sense, mark them as such.
2. extend developer's guide with a description how to run webui tests. A single section or a list of steps will do.
3. look at running those tests as part of CI.
The last step can easily be moved to a separate ticket.0.13Tomek MrugalskiTomek Mrugalskihttps://gitlab.isc.org/isc-projects/stork/-/issues/207add deploying stork packages to cloudsmith.io2020-03-22T09:40:45ZMichal Nowikowskiadd deploying stork packages to cloudsmith.ioThere should be defined testing and public repos. There should be 2 manual GitLab CI jobs:
1) for uploading to testing repo and verifying if pkgs can be installed from that repo
2) the same but for public repoThere should be defined testing and public repos. There should be 2 manual GitLab CI jobs:
1) for uploading to testing repo and verifying if pkgs can be installed from that repo
2) the same but for public repo0.6Michal NowikowskiMichal Nowikowskihttps://gitlab.isc.org/isc-projects/stork/-/issues/208prepare testing pkgs with some functional tests2020-07-21T13:58:37ZMichal Nowikowskiprepare testing pkgs with some functional tests1) install packages
2) invoke some API calls
LXC or LXD containers could be used for that1) install packages
2) invoke some API calls
LXC or LXD containers could be used for that0.10Wlodzimierz WencelWlodzimierz Wencelhttps://gitlab.isc.org/isc-projects/stork/-/issues/383improve CI speed: use `needs` to dependencies between jobs2020-08-31T10:02:20ZMichal Nowikowskiimprove CI speed: use `needs` to dependencies between jobsmore here: https://gitlab.isc.org/help/ci/yaml/README.md#needsmore here: https://gitlab.isc.org/help/ci/yaml/README.md#needs0.11Michal NowikowskiMichal Nowikowskihttps://gitlab.isc.org/isc-projects/stork/-/issues/399automate tarballs and release notes deployment2020-09-14T07:06:43ZMichal Nowikowskiautomate tarballs and release notes deployment0.12https://gitlab.isc.org/isc-projects/stork/-/issues/400problems with access to private registry in push operation in demo deployment2020-11-04T12:40:11ZMichal Nowikowskiproblems with access to private registry in push operation in demo deployment0.13Michal NowikowskiMichal Nowikowskihttps://gitlab.isc.org/isc-projects/stork/-/issues/402automated stork.lab deployment2020-10-15T09:46:55ZTomek Mrugalskiautomated stork.lab deploymentAfter recent problems with deployment, we decided to try more frequent deployment to stork.lab to spot potential problems sooner.
Let's try deploying after every merge to master. If that doesn't work, we'll figure out something else.After recent problems with deployment, we decided to try more frequent deployment to stork.lab to spot potential problems sooner.
Let's try deploying after every merge to master. If that doesn't work, we'll figure out something else.0.13Michal NowikowskiMichal Nowikowskihttps://gitlab.isc.org/isc-projects/stork/-/issues/407automate combining ChangeLog into release notes2021-03-02T18:22:51ZMichal Nowikowskiautomate combining ChangeLog into release notesoutstandinghttps://gitlab.isc.org/isc-projects/stork/-/issues/442deploy_demo job fails on copying agent-kea-many-subnets2020-11-04T14:47:07ZMarcin Siodelskideploy_demo job fails on copying agent-kea-many-subnetsThe deploy_demo CI job fails. See: https://gitlab.isc.org/isc-projects/stork/-/jobs/1272627The deploy_demo CI job fails. See: https://gitlab.isc.org/isc-projects/stork/-/jobs/12726270.13Michal NowikowskiMichal Nowikowskihttps://gitlab.isc.org/isc-projects/stork/-/issues/496Many warnings when running ci_ui2022-10-25T11:59:03ZTomek MrugalskiMany warnings when running ci_uiWhile the `ci_ui` checks [pass now](https://gitlab.isc.org/isc-projects/stork/-/jobs/1543321), there are many warning such as these:
```
WARN: ''app-help-tip' is not a known element:
1. If 'app-help-tip' is an Angular component, then ve...While the `ci_ui` checks [pass now](https://gitlab.isc.org/isc-projects/stork/-/jobs/1543321), there are many warning such as these:
```
WARN: ''app-help-tip' is not a known element:
1. If 'app-help-tip' is an Angular component, then verify that it is part of this module.
2. If 'app-help-tip' is a Web Component then add 'CUSTOM_ELEMENTS_SCHEMA' to the '@NgModule.schemas' of this component to
```
There are warnings about app-breadcrumbs, app-settings-menu, p-panel, p-tabMenu, p-table, app-settings-menu, p-dialog, p-footer, app-help-tip, app-rename-app-dialog, app-events-panel, p-paginator, p-overlayPanel, p-menu, p-button.
Seems we're missing a lot of imports somewhere.outstandinghttps://gitlab.isc.org/isc-projects/stork/-/issues/553deploy_demo complains about docker compose version mismatch2021-08-09T07:24:13ZAndrei Pavelandrei@isc.orgdeploy_demo complains about docker compose version mismatchdeploy_demo doesn't work at all currently. The deployed demo is version 1.5.
https://gitlab.isc.org/isc-projects/stork/-/jobs/1771293
```
Version mismatch: file ./deploy/docker-compose.yaml specifies version 2.1 but extension file ./de...deploy_demo doesn't work at all currently. The deployed demo is version 1.5.
https://gitlab.isc.org/isc-projects/stork/-/jobs/1771293
```
Version mismatch: file ./deploy/docker-compose.yaml specifies version 2.1 but extension file ./deploy/docker-compose-premium.yaml uses version 3.0
```0.19https://gitlab.isc.org/isc-projects/stork/-/issues/587IPv6 support in Stork system test framework2023-07-27T12:26:19ZSlawek FigielIPv6 support in Stork system test frameworkThe system tests framework incorrectly handles IPv6 addresses. It causes a system test building to fail on some configurations.
We need to rewrite our scripts to support this IP schema.
We noticed also that some system tests were execut...The system tests framework incorrectly handles IPv6 addresses. It causes a system test building to fail on some configurations.
We need to rewrite our scripts to support this IP schema.
We noticed also that some system tests were executed, but produce incorrect results when IPv6 was used. We need to prepare some unit and system tests to cover IPv6-based configurations.
```
distro_agent = 'ubuntu/18.04', distro_server = 'centos/8'
@pytest.mark.parametrize("distro_agent, distro_server", SUPPORTED_DISTROS)
def test_pkg_upgrade_server_token(distro_agent, distro_server):
"""Check if Stork agent and server can be upgraded from latest release
to localy built packages."""
server = containers.StorkServerContainer(alias=distro_server)
agent = containers.StorkAgentContainer(alias=distro_agent)
# install the latest version of stork from cloudsmith
server.setup_bg('cloudsmith')
while server.mgmt_ip is None:
time.sleep(0.1)
agent.setup_bg('cloudsmith', server.mgmt_ip)
server.setup_wait()
agent.setup_wait()
# login
r = server.api_post('/sessions', json=dict(useremail='admin', userpassword='admin'), expected_status=200) # TODO: POST should return 201
assert r.json()['login'] == 'admin'
# install local packages
banner('UPGRADING STORK SERVER')
server.prepare_stork_server()
# get server token from server
for i in range(100):
try:
r = server.api_get('/machines-server-token')
break
except:
if i == 99:
raise
time.sleep(1)
data = r.json()
server_token = data['token']
# install kea on the agent machine
agent.install_kea()
# install local packages using server token based way
banner('UPGRADING STORK AGENT')
server_url = 'http://%s:8080' % server.mgmt_ip
> agent.run('curl -o stork-install-agent.sh %s/stork-install-agent.sh' % server_url)
agent = <containers.StorkAgentContainer object at 0x7fdd88d29970>
data = {'token': 'o85LV4YpOCqapgfaiZ4feeoUJL4fiw8v'}
distro_agent = 'ubuntu/18.04'
distro_server = 'centos/8'
i = 0
r = <Response [200]>
server = <containers.StorkServerContainer object at 0x7fdd88bfa040>
server_token = 'o85LV4YpOCqapgfaiZ4feeoUJL4fiw8v'
server_url = 'http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080'
tests.py:211:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <containers.StorkAgentContainer object at 0x7fdd88d29970>, cmd = 'curl -o stork-install-agent.sh http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080/stork-install-agent.sh'
env = {'LANG': 'en_US.UTF-8', 'LANGUAGE': 'en_US:UTF-8', 'LC_ALL': 'en_US.UTF-8'}, ignore_error = False, attempts = 1, sleep_time_after_attempt = None
def run(self, cmd, env=None, ignore_error=False, attempts=1, sleep_time_after_attempt=None):
cmd2 = shlex.split(cmd)
if env is None:
env = {}
env['LANG'] = "en_US.UTF-8"
env['LANGUAGE'] = "en_US:UTF-8"
env['LC_ALL'] = "en_US.UTF-8"
for attempt in range(attempts):
result = self.cntr.execute(cmd2, env)
out = 'run: %s\n' % cmd
out += result[1]
self._trace_logs(out, 'out')
self._trace_logs(result[2], 'err')
if result[0] == 0:
break
elif attempt < attempts - 1:
print('command failed, retry, attempt %d/%d' % (attempt, attempts))
if sleep_time_after_attempt:
time.sleep(sleep_time_after_attempt)
if result[0] != 0 and not ignore_error:
> raise Exception('problem with cmd: %s' % cmd)
E Exception: problem with cmd: curl -o stork-install-agent.sh http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080/stork-install-agent.sh
attempt = 0
attempts = 1
cmd = 'curl -o stork-install-agent.sh http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080/stork-install-agent.sh'
cmd2 = ['curl', '-o', 'stork-install-agent.sh', 'http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080/stork-install-agent.sh']
env = {'LANG': 'en_US.UTF-8', 'LANGUAGE': 'en_US:UTF-8', 'LC_ALL': 'en_US.UTF-8'}
ignore_error = False
out = 'run: curl -o stork-install-agent.sh http://fd42:6657:6f41:ab43:216:3eff:fe59:2fea:8080/stork-install-agent.sh\n'
result = InstanceExecuteResult(exit_code=3, stdout='', stderr="curl: (3) Port number ended with ':'\n")
self = <containers.StorkAgentContainer object at 0x7fdd88d29970>
sleep_time_after_attempt = None
containers.py:228: Exception
```outstandinghttps://gitlab.isc.org/isc-projects/stork/-/issues/597CI: solve TODO about automating changelog update in release notes2022-10-05T14:46:17ZAndrei Pavelandrei@isc.orgCI: solve TODO about automating changelog update in release notes```
# TODO:
# - automate pasting ChangeLog.md to release notes
``````
# TODO:
# - automate pasting ChangeLog.md to release notes
```outstandinghttps://gitlab.isc.org/isc-projects/stork/-/issues/612rake system_tests freezes in CI2021-11-09T16:32:58ZAndrei Pavelandrei@isc.orgrake system_tests freezes in CITimes out at 5 mintues.
```
tests.py::test_users_management[ubuntu/18.04-centos/8]
************ START tests.py::test_users_management[ubuntu/18.04-centos/8] **************************************************************
FAILED
******...Times out at 5 mintues.
```
tests.py::test_users_management[ubuntu/18.04-centos/8]
************ START tests.py::test_users_management[ubuntu/18.04-centos/8] **************************************************************
FAILED
************ RESULT FAILED tests.py::test_users_management[ubuntu/18.04-centos/8] took 0:05:00 ****************************************
```1.0https://gitlab.isc.org/isc-projects/stork/-/issues/615access denied in deploy_demo2021-12-02T13:17:20ZAndrei Pavelandrei@isc.orgaccess denied in deploy_demohttps://stork.lab.isc.org/ has been offline for a while because of the following error in `deploy_demo`:
https://gitlab.isc.org/isc-projects/stork/-/jobs/2095564
```
Pushing agent-kea-premium (registry.gitlab.isc.org/isc-private/stork/...https://stork.lab.isc.org/ has been offline for a while because of the following error in `deploy_demo`:
https://gitlab.isc.org/isc-projects/stork/-/jobs/2095564
```
Pushing agent-kea-premium (registry.gitlab.isc.org/isc-private/stork/agent-kea-premium:latest)...
The push refers to repository [registry.gitlab.isc.org/isc-private/stork/agent-kea-premium]
denied: requested access to the resource is denied
rake aborted!
Command failed with status (1): [docker-compose -f docker-compose.yaml -f d...]
/builds/isc-projects/stork/Rakefile:672:in `block in <top (required)>'
Tasks: TOP => build_and_push_demo_images
```1.0https://gitlab.isc.org/isc-projects/stork/-/issues/686deploy_demo deploys old version of stork2022-02-02T13:58:59ZAndrei Pavelandrei@isc.orgdeploy_demo deploys old version of storkOn release of 1.1.0, the login page displayed version 1.0.0, showing a problem with what version is being deployed in the demo.On release of 1.1.0, the login page displayed version 1.0.0, showing a problem with what version is being deployed in the demo.1.1Andrei Pavelandrei@isc.orgAndrei Pavelandrei@isc.orghttps://gitlab.isc.org/isc-projects/stork/-/issues/702Deploy gosec security checker in CI (SAST)2022-03-02T11:53:13ZTomek MrugalskiDeploy gosec security checker in CI (SAST)There's [gosec](https://github.com/securego/gosec) tool that supposedly is useful for go code audit. We should:
1. [ ] try it and see if the results produced are useful
2. [ ] fix the problems it reported
3. [ ] deploy it on CI
Each st...There's [gosec](https://github.com/securego/gosec) tool that supposedly is useful for go code audit. We should:
1. [ ] try it and see if the results produced are useful
2. [ ] fix the problems it reported
3. [ ] deploy it on CI
Each step depends on the previous one. If at any step we decide the whole thing doesn't make sense, the ticket should be closed.
It may be integrated with gitlab. Go to Security & Compliance -> Configuration, then Static Application Security Testing (SAST).1.3Marcin SiodelskiMarcin Siodelski