stork issueshttps://gitlab.isc.org/isc-projects/stork/-/issues2024-03-25T17:24:07Zhttps://gitlab.isc.org/isc-projects/stork/-/issues/1335Not possible to run DHCPv6 only and collect stats with Prometheus2024-03-25T17:24:07ZRinse KloekNot possible to run DHCPv6 only and collect stats with Prometheus - Kea version: 2.4.1
- Stork: which version? 1.15.0
- OS: Debian 11
I have an issue with the prometheus exporter. I had enabled DHCPv4/DHCPv6/D2 in my kea-ctrl-agent as well as installed the KEA DHCPv4 server. However this server on... - Kea version: 2.4.1
- Stork: which version? 1.15.0
- OS: Debian 11
I have an issue with the prometheus exporter. I had enabled DHCPv4/DHCPv6/D2 in my kea-ctrl-agent as well as installed the KEA DHCPv4 server. However this server only serves DHCPv6 request. If I remove the DHCPv6/D2 entries in the Kea-ctrl-agent.conf and deinstall KEA DHCPv4 server, I am not able to see my DHCPv6 PD stats (kea_dhcp6_pd_assigned_total) anymore.
I get this error message in the log on the server
stork-agent[929757]: time="2024-03-19 11:54:23" level="error" msg="Problem parsing DHCPv4 labels from Kea: problem with content of DHCP labels response from Kea: forwarding socket is not configured for the server type dhcp4\nisc.org/stork/agent.(*SubnetList).UnmarshalJSON\n\t/builds/isc-projects/stork/backend/agent/promkeaexporter.go:82\nencoding/json.(*decodeState).array\n\t/builds/isc-projects/stork/tools/golang/go/src/encoding/json/decode.go:507\nencoding/json.(*decodeState).value\n\t/builds/isc-projects/stork/tools/golang/go/src/encoding/json/decode.go:364\nencoding/json.(*decodeState).unmarshal\n\t/builds/isc-projects/stork/tools/golang/go/src/encoding/json/decode.go:181\nencoding/json.Unmarshal\n\t/builds/isc-projects/stork/tools/golang/go/src/encoding/json/decode.go:108\nisc.org/stork/agent.(*lazySubnetNameLookup).fetchAndCacheNames\n\t/builds/isc-projects/stork/backend/agent/promkeaexporter.go:274\nisc.org/stork/agent.(*lazySubnetNameLookup).getName\n\t/builds/isc-projects/stork/backend/agent/promkeaexporter.go:291\nisc.org/stork/agent.(*lazySubnetNameLookup).getNameOrDefault\n\t/builds/isc-projects/stork/backend/agent/promkeaexporter.go:303\nisc.org/stork/agent.(*PromKeaExporter).setDaemonStats\n\t/builds/isc-projects/stork/backend/agent/promkeaexporter.go:772\nisc.org/stork/agent.(*PromKeaExporter).collectStats\n\t/builds/isc-projects/stork/backend/agent/promkeaexporter.go:877\nisc.org/stork/agent.(*PromKeaExporter).statsCollectorLoop\n\t/builds/isc-projects/stork/backend/agent/promkeaexporter.go:724\nruntime.goexit\n\t/builds/isc-projects/stork/tools/golang/go/src/runtime/asm_amd64.s:1650" file=" promkeaexporter.go:276 "
`
After installing the KEA DHCPv4 server again and reeneabling the KEA DHCPv4 server in kea-ctrl-agent these log message don't appear anymore and the Prometheus stats collection works again.
Is it possible to remove the DHCPv4 server and keep the stats working for DHCPv6
regads,
Rinsehttps://gitlab.isc.org/isc-projects/stork/-/issues/1193CA returns number overflow on ipv6 stats2024-03-22T12:56:27ZSlawek FigielCA returns number overflow on ipv6 statsThe issue was found during [1.13 sanity checks](https://gitlab.isc.org/isc-projects/stork/-/issues/1187#note_408666) by @slawek.
I have observed a weird error in logs:
```
stork-1130-agent-kea6-1 | INFO COMMAND_RECEIVED R...The issue was found during [1.13 sanity checks](https://gitlab.isc.org/isc-projects/stork/-/issues/1187#note_408666) by @slawek.
I have observed a weird error in logs:
```
stork-1130-agent-kea6-1 | INFO COMMAND_RECEIVED Received command 'statistic-get-all'
stork-1130-agent-kea6-1 | INFO CTRL_AGENT_COMMAND_RECEIVED command statistic-get-all received from remote address 127.0.0.1
stork-1130-agent-kea6-1 | INFO COMMAND_RECEIVED Received command 'statistic-get-all'
stork-1130-agent-kea6-1 | time="2023-10-10 12:40:51" level="error" msg="Failed to parse responses from Kea: response result from Kea != 0: 1, text: internal server error: unable to parse server's answer to the forwarded message: Number overflow: 36893488147419103232 in <wire>:0:8422" file=" promkeaexporter.go:850 "
stork-1130-agent-kea6-1 | time="2023-10-10 12:40:51" level="error" msg="Some errors were encountered while collecting stats from Kea: response result from Kea != 0: 1, text: internal server error: unable to parse server's answer to the forwarded message: Number overflow: 36893488147419103232 in <wire>:0:8422\nisc.org/stork/agent.(*GetAllStatisticsResponse).UnmarshalJSON\n\tisc.org/stork/agent/promkeaexporter.go:149\nencoding/json.(*decodeState).array\n\tencoding/json/decode.go:507\nencoding/json.(*decodeState).value\n\tencoding/json/decode.go:364\nencoding/json.(*decodeState).unmarshal\n\tencoding/json/decode.go:181\nencoding/json.Unmarshal\n\tencoding/json/decode.go:108\nisc.org/stork/agent.(*PromKeaExporter).collectStats\n\tisc.org/stork/agent/promkeaexporter.go:847\nisc.org/stork/agent.(*PromKeaExporter).statsCollectorLoop\n\tisc.org/stork/agent/promkeaexporter.go:710\nruntime.goexit\n\truntime/asm_amd64.s:1650" file=" promkeaexporter.go:712 "
```1.16Slawek FigielSlawek Figiel2024-05-29https://gitlab.isc.org/isc-projects/stork/-/issues/1322Upgrade Grafana and its dashboards2024-03-05T15:02:12ZSlawek FigielUpgrade Grafana and its dashboardsYou cannot import the example Grafana dashboards to a modern Grafana instance.
![image](/uploads/a95f33a1aa876ae18d92057fd933fc5f/image.png)
The Stork demo uses the `8.3.7` Grafana version. The latest version is `10.3.3`.
We should up...You cannot import the example Grafana dashboards to a modern Grafana instance.
![image](/uploads/a95f33a1aa876ae18d92057fd933fc5f/image.png)
The Stork demo uses the `8.3.7` Grafana version. The latest version is `10.3.3`.
We should upgrade the Grafana (and maybe Prometheus) used in the demo and migrate the example dashboards to a modern format.1.16https://gitlab.isc.org/isc-projects/stork/-/issues/1153add pool statistics to prometheus2023-09-12T13:48:16ZRazvan Becheriuadd pool statistics to prometheusnew stats have been added in 2.4.0:
v4:
```
"subnet[1].pool[0].assigned-addresses": [
[
0,
"2023-06-13 20:42:46.836205"
]
],
"subnet[1...new stats have been added in 2.4.0:
v4:
```
"subnet[1].pool[0].assigned-addresses": [
[
0,
"2023-06-13 20:42:46.836205"
]
],
"subnet[1].pool[0].cumulative-assigned-addresses": [
[
0,
"2023-06-13 20:42:46.836137"
]
],
"subnet[1].pool[0].declined-addresses": [
[
0,
"2023-06-13 20:42:46.836213"
]
],
"subnet[1].pool[0].reclaimed-declined-addresses": [
[
0,
"2023-06-13 20:42:46.836225"
]
],
"subnet[1].pool[0].reclaimed-leases": [
[
0,
"2023-06-13 20:42:46.836236"
]
],
"subnet[1].pool[0].total-addresses": [
[
11010049,
"2023-06-13 20:42:46.836128"
]
],
```
v6:
```
"subnet[1].pd-pool[0].assigned-pds": [
[
0,
"2023-06-13 21:28:57.196785"
]
],
"subnet[1].pd-pool[0].cumulative-assigned-pds": [
[
0,
"2023-06-13 21:28:57.196744"
]
],
"subnet[1].pd-pool[0].reclaimed-leases": [
[
0,
"2023-06-13 21:28:57.196789"
]
],
"subnet[1].pd-pool[0].total-pds": [
[
256,
"2023-06-13 21:28:57.196741"
]
],
"subnet[1].pool[0].assigned-nas": [
[
0,
"2023-06-13 21:28:57.196773"
]
],
"subnet[1].pool[0].cumulative-assigned-nas": [
[
0,
"2023-06-13 21:28:57.196739"
]
],
"subnet[1].pool[0].declined-addresses": [
[
0,
"2023-06-13 21:28:57.196775"
]
],
"subnet[1].pool[0].reclaimed-declined-addresses": [
[
0,
"2023-06-13 21:28:57.196779"
]
],
"subnet[1].pool[0].reclaimed-leases": [
[
0,
"2023-06-13 21:28:57.196783"
]
],
"subnet[1].pool[0].total-nas": [
[
281474976710656,
"2023-06-13 21:28:57.196736"
]
],
```
```backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/839stork-agent: Encountered unsupported stat2022-12-05T10:00:11ZMunroe Sollogstork-agent: Encountered unsupported stat---
name: Bug report
about: stork-agent Encountered unsupported stat
---
**Describe the bug**
When enabling prometheus output I get errors in the log and no data in grafana:
```
Aug 16 10:43:39 rover stork-agent[56760]: time="2022-08-1...---
name: Bug report
about: stork-agent Encountered unsupported stat
---
**Describe the bug**
When enabling prometheus output I get errors in the log and no data in grafana:
```
Aug 16 10:43:39 rover stork-agent[56760]: time="2022-08-16 10:43:39" level="info" msg="Encountered unsupported stat: cumulative-assigned-addresses" file=" promkeaexporter.go:683 "
Aug 16 10:43:39 rover stork-agent[56760]: time="2022-08-16 10:43:39" level="info" msg="Encountered unsupported stat: reclaimed-declined-addresses" file=" promkeaexporter.go:683 "
Aug 16 10:43:39 rover stork-agent[56760]: time="2022-08-16 10:43:39" level="info" msg="Encountered unsupported stat: declined-addresses" file=" promkeaexporter.go:683 "
Aug 16 10:43:39 rover stork-agent[56760]: time="2022-08-16 10:43:39" level="info" msg="Encountered unsupported stat: reclaimed-leases" file=" promkeaexporter.go:683 "
```
**To Reproduce**
Steps to reproduce the behavior:
Stork 1.5.0 with config:
```
STORK_AGENT_LISTEN_PROMETHEUS_ONLY=true
STORK_AGENT_PROMETHEUS_KEA_EXPORTER_ADDRESS= 127.0.0.1
STORK_AGENT_PROMETHEUS_KEA_EXPORTER_INTERVAL= 60
STORK_AGENT_PROMETHEUS_KEA_EXPORTER_PER_SUBNET_STATS=true
STORK_AGENT_SKIP_TLS_CERT_VERIFICATION=true
```
**Expected behavior**
Data should be be properly scraped from the kea api and exposed via prometheus.
**Environment:**
Kea version info:
```
# kea-dhcp4 -V
2.0.1
tarball
linked with:
log4cplus 1.1.2
OpenSSL 1.1.1n 15 Mar 2022
database:
MySQL backend 12.0, library 10.3.34
PostgreSQL backend 6.2, library 110016
Memfile backend 2.1
```
Using the following hooks:
```
"hooks-libraries": [
{
"library": "/usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_lease_cmds.so",
"parameters": { }
},
{
"library": "/usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_stat_cmds.so",
"parameters": { }
},
{
"library": "/usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_ha.so",
"parameters": {
"high-availability": [
{
"this-server-name": "rover",
"mode": "hot-standby",
"heartbeat-delay": 2000,
"max-response-delay": 12000,
"max-ack-delay": 10000,
"max-unacked-clients": 0,
"multi-threading": {
"enable-multi-threading": true,
"http-dedicated-listener": true,
},
"peers": [
{
"name": "rover",
"role": "primary",
"auto-failover": true
}, {
"name": "spot",
"role": "standby",
"auto-failover": true
}
]
}
]
}
}
]
```1.8Slawek FigielSlawek Figielhttps://gitlab.isc.org/isc-projects/stork/-/issues/576alerting: server should report statistics to Prometheus2021-11-04T10:18:55ZTomek Mrugalskialerting: server should report statistics to PrometheusImplement the ability in Stork server to export statistics to Prometheus. For the time being I think the following stats should be made available:
1. number of authorized machines
2. number of unauthorized machines
3. number of unreacha...Implement the ability in Stork server to export statistics to Prometheus. For the time being I think the following stats should be made available:
1. number of authorized machines
2. number of unauthorized machines
3. number of unreachable machines
The third statistic should report 0 under normal circumstances, so not exactly exciting. The idea is that an admin could set up an alarm for non-zero value reported. For the first two, a careful admin may set up an alert if there's a new machine in the system, e.g. when someone tries to set up a fake machine or when the system is scaled up by adding new VM with Kea.
In the future we may also report the number of error events or something similar. The general idea here is to offload the alterting to prometheus. We were supposed to provide some alterting mechanisms, but instead of implementing them ourselves in Stork, it's much more robust to use existing mechanisms from Prometheus.0.22Slawek FigielSlawek Figielhttps://gitlab.isc.org/isc-projects/stork/-/issues/574No information about the network in the prometheus exporter stork agent2021-11-03T13:46:34ZOleg ZeibelNo information about the network in the prometheus exporter stork agent---
name: Feature request
about: In the prometheus exporter of the agent, you need to show information about the network
---
**Some initial questions**
- Are you sure what you would like to do is not possible using some other mechanisms...---
name: Feature request
about: In the prometheus exporter of the agent, you need to show information about the network
---
**Some initial questions**
- Are you sure what you would like to do is not possible using some other mechanisms?
I have not found how to do it differently
- Stork is in very early stages of development. If your request is not simple, it
may be a while until anyone does anything with your request. Are you ok with that?
Yes
**Is your feature request related to a problem? Please describe.**
Stork prometheus exporter does not provide any information about the DHCP network. Network ID only.
This id has a numerical value and in the graphana dashboard this indicator is not at all visual.
**Describe the solution you'd like**
I would like to see information about the network. For example, in this form
kea_dhcp4_addresses_assigned_total{subnet="10.10.10.0/24"} 3
kea_dhcp4_addresses_declined_reclaimed_total{subnet="10.10.10.0/24"} 0
and other
**Describe alternatives you've considered**
Now we are trying to display the network through the grafana dashboard and the selection of the network ID
**Contacting you**
e-mail: iceadler@gmail.com / o.zejbel@robo.finance0.22Slawek FigielSlawek Figielhttps://gitlab.isc.org/isc-projects/stork/-/issues/341Self-documenting stats (--list-stats switch for agent)2021-06-01T08:44:42ZTomek MrugalskiSelf-documenting stats (--list-stats switch for agent)@matthijs and @tomek discussed the need to have a good statistics list for BIND 9. One way to achieve that would be to implement a `--list-stats` kind of command line switch. It would export the list of currently supported stats by the a...@matthijs and @tomek discussed the need to have a good statistics list for BIND 9. One way to achieve that would be to implement a `--list-stats` kind of command line switch. It would export the list of currently supported stats by the agent, along with some reasonably easy to understand, non-cryptic, non-overly abbreviated description.
For bonus points, that list could be in a format that could be integreated into Stork ARM.
This would address a real issue many BIND 9 users are having: BIND 9 has lots of stats, but nobody really knows what they mean. We won't solve the problem instantly. Incremental steps are the way to go.
We want to have something similar for Kea. However, Kea stats are well documented, so there is less pressing need for this (which also makes the issue easier to solve for Kea, by employing well proven copy-paste programming techniques) ;)backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/231Links to Grafana2020-04-16T19:55:33ZTomek MrugalskiLinks to GrafanaWe need a better intergration with Grafana. For the time being we could add the following:
- [x] ~~ability to open Grafana in iframe. This is lightweight and would look as visually integrated solution. (I've attached a mockup of how thi...We need a better intergration with Grafana. For the time being we could add the following:
- [x] ~~ability to open Grafana in iframe. This is lightweight and would look as visually integrated solution. (I've attached a mockup of how this could look like).~~ @godfryd tried this and it was infeasible due to XSS protections. WE opted to go with dedicated links on stork pages.
- [ ] for the demo in April, the link could be hardcoded
- [x] for the 0.7 release, we could have the Grafana (and Prometheus) links configurable in the db.0.7Michal NowikowskiMichal Nowikowskihttps://gitlab.isc.org/isc-projects/stork/-/issues/175Agent shouldn't open stats exporter port for Kea when Kea is not detected.2020-03-03T16:54:07ZTomek MrugalskiAgent shouldn't open stats exporter port for Kea when Kea is not detected.This is a follow-up to https://gitlab.isc.org/isc-projects/stork/merge_requests/77#note_112821.
The problem is that the agent starts Kea exporter even when there's no Kea detected at all.
This has couple problems:
- unnecessary CPU cyc...This is a follow-up to https://gitlab.isc.org/isc-projects/stork/merge_requests/77#note_112821.
The problem is that the agent starts Kea exporter even when there's no Kea detected at all.
This has couple problems:
- unnecessary CPU cycles (minor)
- opened port when it's not needed. This may be an attack vector.outstandinghttps://gitlab.isc.org/isc-projects/stork/-/issues/174Kea stats list should not be hardcoded in the agent2020-03-03T16:52:28ZTomek MrugalskiKea stats list should not be hardcoded in the agentThe !77 introduced a nice feature: the stork agent extracts Kea statistics and exposes them in Prometheus format.
However, the list of stats is hardcoded. This should be more flexible. My proposal is to store the list in a text file.
Up...The !77 introduced a nice feature: the stork agent extracts Kea statistics and exposes them in Prometheus format.
However, the list of stats is hardcoded. This should be more flexible. My proposal is to store the list in a text file.
Upon first start, the agent would look if there's a file for that. If the file isn't there, it would be created. That way agent deployment still would be easy (just copy one binary), but would be flexible at the same time.
This doesn't have to be implemented that way. The goal is to be able to modify the list of statistics to be exported without recompiling the code.outstandinghttps://gitlab.isc.org/isc-projects/stork/-/issues/167integrate stork with prometheus2020-03-02T13:46:24ZMichal Nowikowskiintegrate stork with prometheus0.5Michal NowikowskiMichal Nowikowski