Skip to content

[9.16] Adjust the respdiff failure threshold for a new dataset

Nicki Křížek requested to merge tkrizek/respdiff-threshold-9.16 into bind-9.16

This is just a slight tweak for the respdiff CI test. The new dataset has a different set of queries and it results in a slightly more SERVFAILs rather than timeouts in the respdiff-long-third-party test.

In our comparison script, timeouts are not counted towards the threshold. While the total number of differences remains roughly the same, the different distributions of them (among SERVFAIL vs timeout) warrants a slight bump in the threshold in order to avoid test failures.

Related isc-private/bind-qa!65

sample run: https://gitlab.isc.org/isc-projects/bind9/-/pipelines/139956

Merge request reports