stork issueshttps://gitlab.isc.org/isc-projects/stork/-/issues2020-03-24T16:58:01Zhttps://gitlab.isc.org/isc-projects/stork/-/issues/8Test BIND exporter2020-03-24T16:58:01ZTomek MrugalskiTest BIND exporterThere is existing BIND exporter: https://github.com/digitalocean/bind_exporter
The goal of this task is to try to set it up and evaluate its usefulness.
Take the following aspects into consideration:
- how tricky it is to expand it to...There is existing BIND exporter: https://github.com/digitalocean/bind_exporter
The goal of this task is to try to set it up and evaluate its usefulness.
Take the following aspects into consideration:
- how tricky it is to expand it to report multiple data points (20 or however many kea reports) for each statistic
- what's the license (can we use it)
- how easy it is extend it in general (is it good source base?)
Keep in mind that the goal for now is to make it functional, but in the future we want it to be performant. If the data handling process is not efficient, make notes and possibly propose improvement (but don't write any code yet).https://gitlab.isc.org/isc-projects/stork/-/issues/10Prepare DNS traffic generator2020-04-06T13:22:02ZTomek MrugalskiPrepare DNS traffic generatorUse queryperf, dnsperf, dnsgen or flamethrower (please pick whatever you feel is useful)
The idea is to come up with a traffic that would generate interesting statistics, we don't want to stress BIND too much, just make the traffic vari...Use queryperf, dnsperf, dnsgen or flamethrower (please pick whatever you feel is useful)
The idea is to come up with a traffic that would generate interesting statistics, we don't want to stress BIND too much, just make the traffic varied and statistics that are not just boring zeroes.0.6Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/stork/-/issues/40Req 1.4 - Application software versions2020-01-20T07:35:40ZVicky Riskvicky@isc.orgReq 1.4 - Application software versionsAs an admin, I would like to be able to quickly scan a list of applications and see the Application version, and optionally also a build#. This is so I can tell which machines may need to be updated, or perhaps I want to run multiple dif...As an admin, I would like to be able to quickly scan a list of applications and see the Application version, and optionally also a build#. This is so I can tell which machines may need to be updated, or perhaps I want to run multiple different software versions for diversity.
* Many users will have their own build systems, or use multiple OSes, so the 'version' field may need to include OS, build#, etc & we need to allow for version #s that do not match ISC version numbers in case of OS packages with different numbering systems.
* It could be useful to have a detail view showing config flags the image was built with.
* If possible, it would be ideal to also report the Hooks loaded (Kea hooks, BIND hooks, BIND RPZ plug-in). How do we report the Kea hook versions?Stork-0.4Vicky Riskvicky@isc.orgVicky Riskvicky@isc.orghttps://gitlab.isc.org/isc-projects/stork/-/issues/43Req 1.8 - Dump Service Configuration2022-01-18T14:44:53ZVicky Riskvicky@isc.orgReq 1.8 - Dump Service ConfigurationAs an admin I am able to dump the running configuration of a specified service on a specified server.
I will want to save, open and review the configuration and I may want to email or upload the configuration file to ISC technical suppo...As an admin I am able to dump the running configuration of a specified service on a specified server.
I will want to save, open and review the configuration and I may want to email or upload the configuration file to ISC technical support for assistance.
I am sometimes confused about which configuration file a service is actually running if there is more than one on the server, so please dump the one that is actually running in the service at the time.
This is for debugging purposes. In later releases we would like to be able to parse and display some configuration elements, and to enable the user to alter the configuration and reload it.
Possible details:
I may have more than one of these files from the same or different servers and services, and if they are all named the same thing it won't help me when reviewing them later. If I have used the same filenames on each server (likely) I will need the file names to be more descriptive.
* enable me to specify a filename when saving the file
* possibly have a default filename including the server name and date/time?
* the file can be in the native format of the given service
* I would most often like to save this dump file on the centralized stork server, but if possible I would like to save it on any other network storage available to the stork application.1.0Slawek FigielSlawek Figielhttps://gitlab.isc.org/isc-projects/stork/-/issues/52Req 3.4 - Log viewer2020-08-05T22:50:03ZVicky Riskvicky@isc.orgReq 3.4 - Log viewerAs an administrator I would like to easily view the recent logs for a monitored service. I may do this when I see some indicator of a service degradation as a first step in identifying what the problem may be. This requirement is for a ...As an administrator I would like to easily view the recent logs for a monitored service. I may do this when I see some indicator of a service degradation as a first step in identifying what the problem may be. This requirement is for a simple log viewer. This is envisioned initially as a fairly simple display of the log file on an individual server. This is not a massive database of historical logs with analysis. For extensive log analysis I would be willing to go to a separate application to look at more voluminous historical logs and to get searching and sorting and aggregation.
1. It would be ideal if there is a default location for the log file for that service, if Stork can locate the log file. In any case, when initially setting up Stork to monitor the service I would expect to be prompted to optionally specify the location of the log file(s).
1. I am going to do this infrequently, so it is ok if I have to push a button or something to 'fetch the log' and wait a bit for the screen to populate. I would prefer not to have to leave the stork application to view the log, and after I have 'fetched the log' I would like to be able to click around and look at other information in Stork and come back to the log viewer without having to 'refetch' the log.
1. We expect to only fetch the most recent portion of the log (so -tail of say, the most recent 15 minutes or so).
1. If possible, include platform logs (e.g. platform restarts, OS updates...) and stork application logs.
1. It would be ideal if it is possible to display the last log for a service that has died, but this is 'extra credit.'
1. As far as the user interface, it is fine if the log itself is opened in a new separate window or tab.
1. It is understood that a single application may have multiple log channels, so these will be displayed in separate windows or tabs and not aggregated. 0.10Vicky Riskvicky@isc.orgVicky Riskvicky@isc.orghttps://gitlab.isc.org/isc-projects/stork/-/issues/54Req 5.1 View Zones List2021-06-01T08:44:42ZVicky Riskvicky@isc.orgReq 5.1 View Zones ListAs an administrator I would like to be able to browse a list of DNS zones that I am publishing, along with a bunch of information on the zone.
1. This is likely to be a very large table, with pages of data, so I would like to be able t...As an administrator I would like to be able to browse a list of DNS zones that I am publishing, along with a bunch of information on the zone.
1. This is likely to be a very large table, with pages of data, so I would like to be able to apply filters to make it more manageable.
1. I want to be able to accommodate up to 2M small zones, 2M RRs zone, 100 views.
1. I would like to be able to sort this by zone name, zone type, time of last update (this might be the default sort), zone size? signing status (signed/unsigned/expired?), #RRs.
1. This zone list should include 'dynamic', 'traditional', catalog, automatic, mirror, root hints, forward, stub, static stub zones.
1. I would like to know the zone type and permit filtering based on zone type.
1. I would like to search based on ... (?cnames?)
1. I would like to know which slaves are publishing that zone
1. I may know a zone name, or partial zone name and will want to know more about that zone.backlogMatthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/stork/-/issues/55Req 5.1 - Zone Transfer Impact2021-06-01T08:44:42ZVicky Riskvicky@isc.orgReq 5.1 - Zone Transfer ImpactFrom BIND GL issue #513
As an administrator I need to determine the impact of large zone updates on operations.
I may see a drop in QPS performance and want to investigate whether this was caused by a large zone transfer. I will need t...From BIND GL issue #513
As an administrator I need to determine the impact of large zone updates on operations.
I may see a drop in QPS performance and want to investigate whether this was caused by a large zone transfer. I will need to see information that will help me identify which zone, how large it is, when it was updated, so that I can see if I can adjust the configuration to ameliorate the impact of large zone transfers.
Details
* Add metrics on the size of the IXFRs e.g. min, max and average size of IXFRs
* Add the same details to the XFR log on the master that are reported on the secondary:
* transfer of 'example.com/IN' from 127.0.0.1#7753: Transfer completed: 1 messages, 14 records, 986 bytes, 0.001 secs (986000 bytes/sec). The log on the master currently only reports that the transfer started and ended.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/56Req 5.3 - View Zone Status2021-06-01T08:44:40ZVicky Riskvicky@isc.orgReq 5.3 - View Zone Status"from a user ""It would be very helpful for us to have the various zone timers exposed through the statistics channel. The information is currently available through `rndc zonestatus`, but it would be far easier for us to monitor the ser..."from a user ""It would be very helpful for us to have the various zone timers exposed through the statistics channel. The information is currently available through `rndc zonestatus`, but it would be far easier for us to monitor the servers if this were accessible through the stats channel.
Our use case would be to monitor for zones approaching expiration. We'd like to use the stats channel to pull the full list of zones with the timers in one operation, and then parse the data."""backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/57Req 5.4 - Zone Signing Status2021-06-01T08:44:41ZVicky Riskvicky@isc.orgReq 5.4 - Zone Signing StatusAs an admin I want to see DNSSEC details, key information, signature validity period, when is the next key rollover, when is the next resign, and what is the zone that will be resigned next.. nsec3As an admin I want to see DNSSEC details, key information, signature validity period, when is the next key rollover, when is the next resign, and what is the zone that will be resigned next.. nsec3backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/58Req 5.5 - View NTAs2023-04-11T16:19:44ZVicky Riskvicky@isc.orgReq 5.5 - View NTAsAs an administrator, I need to see what Negative trust anchors are configured. I may have help desk staff that need to be prepared to answer questions about zones that may stop validating.
Questions I have:
* What NTAs are active?
* Fo...As an administrator, I need to see what Negative trust anchors are configured. I may have help desk staff that need to be prepared to answer questions about zones that may stop validating.
Questions I have:
* What NTAs are active?
* For the NTAs configured, when do they expire?
* I also want to see any 'permanent NTAs'. (zones with = validate except.)backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/59Req 5.6 - View Query Activity2020-04-07T15:18:34ZVicky Riskvicky@isc.orgReq 5.6 - View Query ActivityAs an administrator I would like to be able to see the queries per second received and answered per server.
I would like to see a line chart so I can see time of day patterns, and trends, spikes or dips from the 'usual' level of activity...As an administrator I would like to be able to see the queries per second received and answered per server.
I would like to see a line chart so I can see time of day patterns, and trends, spikes or dips from the 'usual' level of activity. This will help identify the best times for maintenance activities (low usage times) and will help in capacity planning.0.7Vicky Riskvicky@isc.orgVicky Riskvicky@isc.orghttps://gitlab.isc.org/isc-projects/stork/-/issues/60Req 5.7 - View RPZ Statistics2021-06-01T08:44:42ZVicky Riskvicky@isc.orgReq 5.7 - View RPZ StatisticsAs an administrator I need to know how much of an impact RPZ is having.
I may be either introducing RPZ for the first time, or trialing an additional RPZ feed and attempting to evaluate how many more matches are found with the addition ...As an administrator I need to know how much of an impact RPZ is having.
I may be either introducing RPZ for the first time, or trialing an additional RPZ feed and attempting to evaluate how many more matches are found with the addition of a new zone(s). I would like to be able to report the number of possible 'bad' queries blocked to management, to justify the cost of commercial RPZ feeds.
The most basic metric is a global counter (eg. 15 minute intervals) of RPZ matches. If we just have a global counter of RPZ matches, then if the user adds a new RPZ feed, they can look to see how much that number changed by.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/61Req 5.7.2 - RPZ Detail2021-06-01T08:44:42ZVicky Riskvicky@isc.orgReq 5.7.2 - RPZ DetailAs a user, I would like to know how many RPZ matches are coming from *each* RPZ zone. RPZ zones are evaluated in order they are configured, so if two zones include the same filter, the 'match' will be attributed to the first RPZ listed. ...As a user, I would like to know how many RPZ matches are coming from *each* RPZ zone. RPZ zones are evaluated in order they are configured, so if two zones include the same filter, the 'match' will be attributed to the first RPZ listed.
If we can communicate this (the order of the RPZ zones and its relationship to how many answers were blocked by each zone) in the UI that would be helpful.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/62Req 5.7.1 - RPZ Response Actions2021-06-01T08:44:41ZVicky Riskvicky@isc.orgReq 5.7.1 - RPZ Response ActionsAs a user, I would like to investigate RPZ matches to determine or estimate the type of abuse being blocked by RPZ.
I can extrapolate the type of abuse (malware, legal filtering, etc) based on the type of RPZ action.
Report statistics ...As a user, I would like to investigate RPZ matches to determine or estimate the type of abuse being blocked by RPZ.
I can extrapolate the type of abuse (malware, legal filtering, etc) based on the type of RPZ action.
Report statistics on the type of RPZ action taken (type of action, rewrites, NXDOMAIN etc.)backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/63Req 6.1 - Query Details2020-07-21T15:16:49ZVicky Riskvicky@isc.orgReq 6.1 - Query DetailsAs a user I would like to monitor the volume of queries and responses. This should be fairly prominent.This is a baseline function that everyone needs. These statistics should be available on a per-server basis from BIND today.
In addi...As a user I would like to monitor the volume of queries and responses. This should be fairly prominent.This is a baseline function that everyone needs. These statistics should be available on a per-server basis from BIND today.
In addition, I would like to be able to see in a detail view the distribution of rrtypes, response codes, query levels by TCP vs UDP, perhaps by some **response size** buckets.
Some of the details are not that interesting, unless you are investigating a specific problem (such as with TCP connections). Many of the response codes and rrtypes will be used infrequently and charts will have a lot of empty bars or columns and a few big ones.
Include queries that are dropped, if possible, perhaps in a display by response code?
It could be useful to see these displayed both per-server and aggregated across clusters of servers (for when you are investigating a possible pattern of abuse queries for example).0.10Matthijs Mekkingmatthijs@isc.orgMatthijs Mekkingmatthijs@isc.orghttps://gitlab.isc.org/isc-projects/stork/-/issues/64Req 6.4 - Cache Hit Ratio2020-06-10T09:28:47ZVicky Riskvicky@isc.orgReq 6.4 - Cache Hit RatioAs a user, I want to know the effectiveness of my cache. Improving the cache hit ratio is one of the best ways I can improve overall throughput.
Eventually I would like to be able to test the impact on cache hit ratio of various settin...As a user, I want to know the effectiveness of my cache. Improving the cache hit ratio is one of the best ways I can improve overall throughput.
Eventually I would like to be able to test the impact on cache hit ratio of various settings, for help in tuning my configuration
% of queries answered from cache (time series)0.6https://gitlab.isc.org/isc-projects/stork/-/issues/65Req 6.5 - Cache Details2021-06-01T08:44:41ZVicky Riskvicky@isc.orgReq 6.5 - Cache DetailsAs a user I would like to see details on what is in the cache in order to determine why the cache hit ratio might be low. The purpose of displaying this data is to help guide me about configuration settings that could improve the cache e...As a user I would like to see details on what is in the cache in order to determine why the cache hit ratio might be low. The purpose of displaying this data is to help guide me about configuration settings that could improve the cache effectiveness.
Useful details would include
* cache size (memory, # of records)
* average ttl of records in cache (perhaps also min and max ttl?)
* breakdown by record type, status (valid vs expired)
* LRU of records pre-fetched
* LRU of records that expired without being re-queried
* top 500(?) records most frequently queried
* cache cleaning (how dirty is the cache)backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/66Req 6.7 - Memory Utilization2021-06-01T08:44:42ZVicky Riskvicky@isc.orgReq 6.7 - Memory UtilizationAs a user, I would like to know what named's current memory allocation being used for.
* If I am running low on available memory, I want to identify possible options for reducing memory consumption with a configuration change.
* Alterna...As a user, I would like to know what named's current memory allocation being used for.
* If I am running low on available memory, I want to identify possible options for reducing memory consumption with a configuration change.
* Alternatively, this will help me identify 'runaway' processes that are eating memory and not freeing it as part of a troubleshooting exercise.
* When I am operating a hybrid server I need to see the amount of memory being used for auth vs recursive functions.
Some of this information may be available by querying the machine rather than the service.
We may want to review what would be presented. As an operator, I am not going to benefit from really cryptic references to processes inside BIND that I cannot control or stop. However, ISC technical support might want some long list of arcane stuff that I cannot interpret.backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/67Req 7.1 - Performance Troubleshooting2021-06-01T08:44:41ZVicky Riskvicky@isc.orgReq 7.1 - Performance TroubleshootingAs a user, I am looking for information that should be flagged that may help understand what is limiting performance currently.
I am particularly concerned about maximizing performance of my resolver.
What are the critical resources I ...As a user, I am looking for information that should be flagged that may help understand what is limiting performance currently.
I am particularly concerned about maximizing performance of my resolver.
What are the critical resources I need to monitor, besides memory?
- [x] CPU
- [ ] threads
- [ ] sockets??
- [ ] TCP connections
- [ ] 'clients'?
what else?
What information is available on what is tying up these resources?
Quote from Cathy "What is BIND doing (while it is, eating memory, eating CPU, not responding, apparently twiddling it's thumbs or ..?)"backloghttps://gitlab.isc.org/isc-projects/stork/-/issues/68Req 7.2 - Throttling2021-06-01T08:44:41ZVicky Riskvicky@isc.orgReq 7.2 - ThrottlingI would like to know if I am throttling traffic based on configured limits. If so, I might want to change these limits to throttle more or less.
These limits are typically designed to protect the system from being overwhelmed in case of...I would like to know if I am throttling traffic based on configured limits. If so, I might want to change these limits to throttle more or less.
These limits are typically designed to protect the system from being overwhelmed in case of a DDOS. However, sometimes the throttles are set low enough that they impact throughput unnecessarily during normal operation.
Priorities
* Fetch-limits
* clients per query
* client-quotas
* TCP quotas
* RRL
? Is this server being throttled by fetch-limits or is this zone being throttled by fetch-limits?
Log instances of crossing the thresholds where throttling kicks in, when you cross the threshold again on the way down.
Several specific problems we would like to address are:
* https://gitlab.isc.org/isc-projects/bind9/issues/665 Add "rndc fetchlimits" command to dump currently-active ADB rate-limited servers and zones
* https://gitlab.isc.org/isc-projects/bind9/issues/915 Add ability to determine frozen zones
* https://gitlab.isc.org/isc-projects/bind9/issues/1232 [ISC-support #15166] expose zone timers (reload, refresh, expire) via stats channelbacklog