GSS-TSIG make GSS_C_REPLAY_FLAG optional
MS DNS servers appear to employ a very strict replay protection algorithm in
the GSS layer (using sequence numbers). From the observable behavior, it seems
to require the sequence number monotonically increase one by one (without any
missing or reordering sequence), and return "broken response" (but
with RCODE being REFUSED) if it detects a violation of this requirement (see #2405).
Further, once this happens any subsequent DDNS requests would result in REFUSED
(in the form of broken response), probably because the request of the "next
expected sequence number" was effectively dropped and never arrives.
This can happen not only because of the real missing or reordered requests, but also (apparently) because of prerequisite failures (in this case, it looks like the MS server doesn't even bother to check the TSIG RR, and returns the broken response). So it can happen quite easily in practice.
So, it simply doesn't make sense to enable the GSS level replay protection with a MS DNS server. If we disabled it by omitting GSS_C_REPLAY_FLAG from TKeyExchange::TKEY_EXCHANGE_FLAGS defined in tkey_exchange.cc, this problem didn't (seem to) happen.
I'd suggest making GSS_C_REPLAY_FLAG optional that can be enabled/disabled in config. Note that TSIG has its own generic (algorithm-independent) replay protection, so even if we disable the GSS-level replay protection, we'd still have some level of defense. And, in any case, this hook is simply unusable with a MS server as long as we include this flag.
see RT #20795