The enginepkcs11 system test is unstable
The enginepkcs11
system test was unavailable for a while, and when its prereq.sh
script got fixed, we realized it is so unstable, that flaky was brought to rescue. But flaky is not enough, the test keeps failing as in, sometimes it won't pass even once in three runs. Today, I saw that the test failed twice on the main
branch.
Job #4232411 failed for f454fa6d.
2024-04-23 11:26:48 INFO:enginepkcs11.test_enginepkcs11 I:Test SOA response for ecdsap256sha256."\:\;\?\&\[\]\@\!$\*\+\,\|\=\.\(\)foo.weird (44)
2024-04-23 11:26:49 INFO:enginepkcs11.test_enginepkcs11 I:failed (expected a SOA RRSIG record)
2024-04-23 11:26:49 INFO:enginepkcs11.test_enginepkcs11 I:Test key generation was successful for ecdsap256sha256.split (44)
2024-04-23 11:26:49 INFO:enginepkcs11.test_enginepkcs11 I:failed (expected 1 key, got 1)
2024-04-23 11:27:00 INFO:enginepkcs11.test_enginepkcs11 I:Test SOA is signed for ecdsap256sha256.same-policy.views in view1 (65)
2024-04-23 11:27:03 INFO:enginepkcs11.test_enginepkcs11 I:failed (SOA RRset not signed)
2024-04-23 11:27:03 INFO:enginepkcs11.test_enginepkcs11 I:Test DNSKEY is signed for ecdsap256sha256.same-policy.views in view1 (66)
2024-04-23 11:27:07 INFO:enginepkcs11.test_enginepkcs11 I:failed (DNSKEY RRset not signed)
2024-04-23 11:27:14 INFO:enginepkcs11.test_enginepkcs11 I:Test key generation was successful for rsasha256.kasp (9)
2024-04-23 11:27:14 INFO:enginepkcs11.test_enginepkcs11 I:failed (expected 2 keys, got 0)
2024-04-23 11:27:14 INFO:enginepkcs11.test_enginepkcs11 cat: 'Krsasha256.kasp*.private': No such file or directory
2024-04-23 11:27:14 INFO:enginepkcs11.test_enginepkcs11 I:failed (expected Label metadata in key files)
2024-04-23 11:27:14 INFO:enginepkcs11.test_enginepkcs11 I:Test DNSKEY response for rsasha256.kasp (10)
2024-04-23 11:27:15 INFO:enginepkcs11.test_enginepkcs11 I:failed (expected 2 DNSKEY records)
2024-04-23 11:27:15 INFO:enginepkcs11.test_enginepkcs11 I:Test SOA response for rsasha256.kasp (11)
2024-04-23 11:27:16 INFO:enginepkcs11.test_enginepkcs11 I:failed (expected a SOA RRSIG record)
2024-04-23 11:27:19 INFO:enginepkcs11.test_enginepkcs11 I:Test SOA is signed for ecdsap256sha256.same-policy.views in view1 (65)
2024-04-23 11:27:22 INFO:enginepkcs11.test_enginepkcs11 I:failed (SOA RRset not signed)
2024-04-23 11:27:22 INFO:enginepkcs11.test_enginepkcs11 I:Test DNSKEY is signed for ecdsap256sha256.same-policy.views in view1 (66)
2024-04-23 11:27:25 INFO:enginepkcs11.test_enginepkcs11 I:failed (DNSKEY RRset not signed)