Stork randomly crashes and then restarts after some time
name: Bug report
about: Stork randomly panics and restarts after some time
Describe the bug
For some unknown reason, Stork will panic and restart after some time.
You can ignore DHCPv6 errors, we don't use it.
I looked at the logs and got these:
2024-05-20T23:34:20.716854+02:00 redacted stork-server[1718583]: panic: runtime error: invalid memory address or nil pointer dereference
2024-05-20T23:34:20.716887+02:00 redacted stork-server[1718583]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xf2cd0e]
2024-05-20T23:34:20.716904+02:00 redacted stork-server[1718583]: goroutine 16 [running]:
2024-05-20T23:34:20.716925+02:00 redacted stork-server[1718583]: isc.org/stork/server/eventcenter.CreateEvent(0x2, {0x1504196, 0x4d}, {0xc000016380, 0x4, 0x7fb62e579e08?})
2024-05-20T23:34:20.716963+02:00 redacted stork-server[1718583]: #011/builds/isc-projects/stork/backend/server/eventcenter/eventcenter.go:121 +0xace
2024-05-20T23:34:20.716977+02:00 redacted stork-server[1718583]: isc.org/stork/server/eventcenter.(*eventCenter).addEvent(0xc0003b8f30, 0xc0008d8390?, {0x1504196?, 0x8a4a65?}, {0xc000016380?, 0x1305480?, 0x1?})
2024-05-20T23:34:20.716996+02:00 redacted stork-server[1718583]: #011/builds/isc-projects/stork/backend/server/eventcenter/eventcenter.go:140 +0x3b
2024-05-20T23:34:20.717013+02:00 redacted stork-server[1718583]: isc.org/stork/server/eventcenter.(*eventCenter).AddErrorEvent(0xc0007b0a80?, {0x1504196?, 0x2?}, {0xc000016380?, 0x0?, 0x0?})
2024-05-20T23:34:20.717036+02:00 redacted stork-server[1718583]: #011/builds/isc-projects/stork/backend/server/eventcenter/eventcenter.go:67 +0x31
2024-05-20T23:34:20.717049+02:00 redacted stork-server[1718583]: isc.org/stork/server/agentcomm.(*connectedAgentsData).ForwardToKeaOverHTTP(0xc000312c80, {0x13a7840?, 0x10?}, {0x16add40, 0xc001538000}, {0xc0007b0940, 0x2, 0x2}, {0xc0007b0920, 0x2, ...})
2024-05-20T23:34:20.717063+02:00 redacted stork-server[1718583]: #011/builds/isc-projects/stork/backend/server/agentcomm/grpcli.go:780 +0x1303
2024-05-20T23:34:20.717076+02:00 redacted stork-server[1718583]: isc.org/stork/server/apps/kea.(*StatsPuller).getStatsFromApp(0xc0000372b0, 0xc001538000)
2024-05-20T23:34:20.717095+02:00 redacted stork-server[1718583]: #011/builds/isc-projects/stork/backend/server/apps/kea/statspuller.go:338 +0x10d4
2024-05-20T23:34:20.717114+02:00 redacted stork-server[1718583]: isc.org/stork/server/apps/kea.(*StatsPuller).pullStats(0xc0000372b0)
2024-05-20T23:34:20.717128+02:00 redacted stork-server[1718583]: #011/builds/isc-projects/stork/backend/server/apps/kea/statspuller.go:63 +0x169
2024-05-20T23:34:20.717149+02:00 redacted stork-server[1718583]: isc.org/stork/server/agentcomm.NewPeriodicPuller.func1()
2024-05-20T23:34:20.717164+02:00 redacted stork-server[1718583]: #011/builds/isc-projects/stork/backend/server/agentcomm/puller.go:42 +0x83
2024-05-20T23:34:20.717174+02:00 redacted stork-server[1718583]: isc.org/stork/util.(*PeriodicExecutor).executorLoop(0xc00030f450)
2024-05-20T23:34:20.717188+02:00 redacted stork-server[1718583]: #011/builds/isc-projects/stork/backend/util/periodicexecutor.go:167 +0xf6
2024-05-20T23:34:20.717200+02:00 redacted stork-server[1718583]: created by isc.org/stork/util.NewPeriodicExecutor in goroutine 1
2024-05-20T23:34:20.717210+02:00 redacted stork-server[1718583]: #011/builds/isc-projects/stork/backend/util/periodicexecutor.go:65 +0x245
Honestly I don't know if its setup issue or some bug in Stork.
Environment:
- Kea version: 2.5.7
- Stork: 1.16.0
- OS: Debian 12
- Kea hooks: MySQL cb, lease cmds, stat cmds and ha
Edited by Vojtěch Dočkal