1. 03 Apr, 2020 22 commits
    • Ondřej Surý's avatar
      Merge branch '1087-fix-the-nonmatching-statcounter-increments-decrements' into 'master' · bb876130
      Ondřej Surý authored
      Fix the some of the underflowing statistics
      
      See merge request !3299
      bb876130
    • Ondřej Surý's avatar
      Add CHANGES · 22aaeb51
      Ondřej Surý authored
      22aaeb51
    • Ondřej Surý's avatar
      Fix the statistic counter underflow in ns_client_t · 78886d4b
      Ondřej Surý authored
      In case of normal fetch, the .recursionquota is attached and
      ns_statscounter_recursclients is incremented when the fetch is created.  Then
      the .recursionquota is detached and the counter decremented in the
      fetch_callback().
      
      In case of prefetch or rpzfetch, the quota is attached, but the counter is not
      incremented.  When we reach the soft-quota, the function returns early but don't
      detach from the quota, and it gets destroyed during the ns_client_endrequest(),
      so no memory was leaked.
      
      But because the ns_statscounter_recursclients is only incremented during the
      normal fetch the counter would be incorrectly decremented on two occassions:
      
      1) When we reached the softquota, because the quota was not properly detached
      2) When the prefetch or rpzfetch was cancelled mid-flight and the callback
         function was never called.
      78886d4b
    • Ondřej Surý's avatar
    • Witold Krecicki's avatar
    • Matthijs Mekking's avatar
      Merge branch '1179-dnssec-stats-oom-kill' into 'master' · 663047ac
      Matthijs Mekking authored
      Resolve "OOM issue after upgrade from 9.14.3 to 9.14.4"
      
      Closes #1179
      
      See merge request !3304
      663047ac
    • Matthijs Mekking's avatar
      Update release notes · 386890a1
      Matthijs Mekking authored
      386890a1
    • Matthijs Mekking's avatar
      Replace hard coded value with constant · c1723b25
      Matthijs Mekking authored
      c1723b25
    • Matthijs Mekking's avatar
      Merge if blocks in statschannel.c · 1596d3b4
      Matthijs Mekking authored
      1596d3b4
    • Matthijs Mekking's avatar
      Replace sign operation bool with enum · 44b49955
      Matthijs Mekking authored
      44b49955
    • Matthijs Mekking's avatar
      Embed algorithm in key tag counter · b2028e26
      Matthijs Mekking authored
      Key tags are not unique across algorithms.
      b2028e26
    • Matthijs Mekking's avatar
      Group the keyid with the counters · eb6a8b47
      Matthijs Mekking authored
      Rather than group key ids together, group key id with its
      corresponding counters. This should make growing / shrinking easier
      than having keyids then counters.
      eb6a8b47
    • Matthijs Mekking's avatar
      Add test for many keys · 31e8b2b1
      Matthijs Mekking authored
      Add a statschannel test case for DNSSEC sign metrics that has more
      keys than there are allocated stats counters for.  This will produce
      gibberish, but at least it should not crash.
      31e8b2b1
    • Matthijs Mekking's avatar
      Redesign dnssec sign statistics · 705810d5
      Matthijs Mekking authored
      The first attempt to add DNSSEC sign statistics was naive: for each
      zone we allocated 64K counters, twice.  In reality each zone has at
      most four keys, so the new approach only has room for four keys per
      zone. If after a rollover more keys have signed the zone, existing
      keys are rotated out.
      
      The DNSSEC sign statistics has three counters per key, so twelve
      counters per zone. First counter is actually a key id, so it is
      clear what key contributed to the metrics.  The second counter
      tracks the number of generated signatures, and the third tracks
      how many of those are refreshes.
      
      This means that in the zone structure we no longer need two separate
      references to DNSSEC sign metrics: both the resign and refresh stats
      are kept in a single dns_stats structure.
      
      Incrementing dnssecsignstats:
      
      Whenever a dnssecsignstat is incremented, we look up the key id
      to see if we already are counting metrics for this key.  If so,
      we update the corresponding operation counter (resign or
      refresh).
      
      If the key is new, store the value in a new counter and increment
      corresponding counter.
      
      If all slots are full, we rotate the keys and overwrite the last
      slot with the new key.
      
      Dumping dnssecsignstats:
      
      Dumping dnssecsignstats is no longer a simple wrapper around
      isc_stats_dump, but uses the same principle.  The difference is that
      rather than dumping the index (key tag) and counter, we have to look
      up the corresponding counter.
      705810d5
    • Matthijs Mekking's avatar
      Merge branch '1706-dnssec-policy-migration' into 'master' · 157eb5cd
      Matthijs Mekking authored
      Resolve "Changing from auto-dnssec maintain to dnssec-policy x immediately deletes existing keys"
      
      Closes #1706
      
      See merge request !3322
      157eb5cd
    • Matthijs Mekking's avatar
      Test migration to dnssec-policy, change algorithm · 551acb44
      Matthijs Mekking authored
      Add a test to ensure migration from 'auto-dnssec maintain;' to
      dnssec-policy works even if the algorithm is changed.  The existing
      keys should not be removed immediately, but their goal should be
      changed to become hidden, and the new keys with the different
      algorithm should be introduced immediately.
      551acb44
    • Matthijs Mekking's avatar
      Only initialize goal on active keys · 2389fcb4
      Matthijs Mekking authored
      If we initialize goals on all keys, superfluous keys that match
      the policy all desire to be active.  For example, there are six
      keys available for a policy that needs just two, we only want to
      set the goal state to OMNIPRESENT on two keys, not six.
      2389fcb4
    • Matthijs Mekking's avatar
      Update documentation with !1706 fix · f47e697d
      Matthijs Mekking authored
      f47e697d
    • Matthijs Mekking's avatar
      Test migration to dnssec-policy, retire old keys · 7f435208
      Matthijs Mekking authored
      Migrating from 'auto-dnssec maintain;' to dnssec-policy did not
      work properly, mainly because the legacy keys were initialized
      badly.  Earlier commit deals with migration where existing keys
      match the policy.  This commit deals with migration where existing
      keys do not match the policy.  In that case, named must not
      immediately delete the existing keys, but gracefully roll to the
      dnssec-policy.
      
      However, named did remove the existing keys immediately.  This is
      because the legacy key states were initialized badly.  Because
      those keys had their states initialized to HIDDEN or RUMOURED, the
      keymgr decides that they can be removed (because only when the key
      has its states in OMNIPRESENT it can be used safely).
      
      The original thought to initialize key states to HIDDEN (and
      RUMOURED to deal with existing keys) was to ensure that those keys
      will go through the required propagation time before the keymgr
      decides they can be used safely.  However, those keys are already
      in the zone for a long time and making the key states represent
      otherwise is dangerous: keys may be pulled out of the zone while
      in fact they are required to establish the chain of trust.
      
      Fix initializing key states for existing keys by looking more closely
      at the time metadata.  Add TTL and propagation delays to the time
      metadata and see if the DNSSEC records have been propagated.
      Initialize the state to OMNIPRESENT if so, otherwise initialize to
      RUMOURED.  If the time metadata is in the future, or does not exist,
      keep initializing the state to HIDDEN.
      
      The added test makes sure that new keys matching the policy are
      introduced, but existing keys are kept in the zone until the new
      keys have been propagated.
      7f435208
    • Matthijs Mekking's avatar
      Tweak kasp system test · a224754d
      Matthijs Mekking authored
      A few kasp system test tweaks to improve test failure debugging and
      deal with tests related to migration to dnssec-policy.
      
      1. When clearing a key, set lifetime to "none".  If "none", skip
         expect no lifetime set in the state file.  Legacy keys that
         are migrated but don't match the dnssec-policy will not have a
         lifetime.
      
      2. The kasp system test prints which key id and file it is checking.
         Log explicitly if we are checking the id or a file.
      
      3. Add quotes around "ID" when setting the key id, for consistency.
      
      4. Fix a typo (non -> none).
      
      5. Print which key ids are found, this way it is easier to see what
         KEY[1-4] failed to match one of the key files.
      a224754d
    • Matthijs Mekking's avatar
      Fix and test migration to dnssec-policy · 68018991
      Matthijs Mekking authored
      Migrating from 'auto-dnssec maintain;' to dnssec-policy did not
      work properly, mainly because the legacy keys were initialized
      badly. Several adjustments in the keymgr are required to get it right:
      
      - Set published time on keys when we calculate prepublication time.
        This is not strictly necessary, but it is weird to have an active
        key without the published time set.
      
      - Initalize key states also before matching keys. Determine the
        target state by looking at existing time metadata: If the time
        data is set and is in the past, it is a hint that the key and
        its corresponding records have been published in the zone already,
        and the state is initialized to RUMOURED. Otherwise, initialize it
        as HIDDEN. This fixes migration to dnssec-policy from existing
        keys.
      
      - Initialize key goal on keys that match key policy to OMNIPRESENT.
        These may be existing legacy keys that are being migrated.
      
      - A key that has its goal to OMNIPRESENT *or* an active key can
        match a kasp key.  The code was changed with CHANGE 5354 that
        was a bugfix to prevent creating new KSK keys for zones in the
        initial stage of signing.  However, this caused problems for
        restarts when rollovers are in progress, because an outroducing
        key can still be an active key.
      
      The test for this introduces a new KEY property 'legacy'.  This is
      used to skip tests related to .state files.
      68018991
    • Ondřej Surý's avatar
      Merge branch '1717-rwlock-contention-in-isc_log_wouldlog-api-performance-impact' into 'master' · 49b42e7f
      Ondřej Surý authored
      Reduce rwlock contention in isc_log_wouldlog()
      
      Closes #1717
      
      See merge request !3321
      49b42e7f
  2. 02 Apr, 2020 1 commit
    • Ondřej Surý's avatar
      Reduce rwlock contention in isc_log_wouldlog() · 3a24eacb
      Ondřej Surý authored
      The rwlock introduced to protect the .logconfig member of isc_log_t
      structure caused a significant performance drop because of the rwlock
      contention.  It was also found, that the debug_level member of said
      structure was not protected from concurrent read/writes.
      
      The .dynamic and .highest_level members of isc_logconfig_t structure
      were actually just cached values pulled from the assigned channels.
      
      We introduced an even higher cache level for .dynamic and .highest_level
      members directly into the isc_log_t structure, so we don't have to
      access the .logconfig member in the isc_log_wouldlog() function.
      3a24eacb
  3. 01 Apr, 2020 3 commits
  4. 31 Mar, 2020 2 commits
  5. 30 Mar, 2020 4 commits
    • Witold Krecicki's avatar
      Merge branch '1700-proper-tcp-resuming' into 'master' · d02a58ad
      Witold Krecicki authored
      Deactivate the handle before sending the async close callback.
      
      Closes #1700
      
      See merge request !3303
      d02a58ad
    • Witold Krecicki's avatar
      Deactivate the handle before sending the async close callback. · 01c4c330
      Witold Krecicki authored
      We could have a race between handle closing and processing async
      callback. Deactivate the handle before issuing the callback - we
      have the socket referenced anyway so it's not a problem.
      01c4c330
    • Witold Krecicki's avatar
      Merge branch 'wpk/quota-callback' into 'master' · 2df610a8
      Witold Krecicki authored
      isc_quota_attach_cb - quota attach function with a callback.
      
      See merge request !3280
      2df610a8
    • Witold Krecicki's avatar
      Add a quota attach function with a callback, some code cleanups. · d151a10f
      Witold Krecicki authored
      We introduce a isc_quota_attach_cb function - if ISC_R_QUOTA is returned
      at the time the function is called, then a callback will be called when
      there's quota available (with quota already attached). The callbacks are
      organized as a LIFO queue in the quota structure.
      It's needed for TCP client quota -  with old networking code we had one
      single place where tcp clients quota was processed so we could resume
      accepting when the we had spare slots, but it's gone with netmgr - now
      we need to notify the listener/accepter that there's quota available so
      that it can resume accepting.
      
      Remove unused isc_quota_force() function.
      
      The isc_quote_reserve and isc_quota_release were used only internally
      from the quota.c and the tests.  We should not expose API we are not
      using.
      d151a10f
  6. 26 Mar, 2020 8 commits