1. 16 Mar, 2022 1 commit
  2. 15 Mar, 2022 2 commits
  3. 14 Mar, 2022 36 commits
    • Ondřej Surý's avatar
      Merge branch '3202-cleanup-isc_timer-API' into 'main' · 13b20ef4
      Ondřej Surý authored
      Refactor and simplify isc_timer API
      
      See merge request !5966
      13b20ef4
    • Ondřej Surý's avatar
      Add CHANGES note for [GL #3202] · 7f91f1ec
      Ondřej Surý authored and Evan Hunt's avatar Evan Hunt committed
      7f91f1ec
    • Ondřej Surý's avatar
      Implement isc_interval_t on top of isc_time_t · 79b5ccbf
      Ondřej Surý authored and Evan Hunt's avatar Evan Hunt committed
      Change the isc_interval_t implementation from separate data type and
      separate implementation to be shim implementation on top of isc_time_t.
      The distinction between isc_interval_t and isc_time_t has been kept
      because they are semantically different - isc_interval_t is relative and
      isc_time_t is absolute, but this allows isc_time_t and isc_interval_t to
      be freely interchangeable, f.e. this:
      
          isc_time_t *t1;
          isc_interval_t *interval;
          isc_time_t *t2;
      
          isc_interval_set(interval, isc_time_seconds(t2), isc_time_nanoseconds(t2);;
          isc_time_subtract(t1, interval, t2);
          isc_interval_set(interval, isc_time_seconds(t2), isc_time_nanoseconds(t2));
      
      to just:
      
          isc_time_t *t1;
          isc_interval_t *interval;
          isc_time_t *t2;
      
          isc_time_subtract(t1, t2, interval);
      
      without introducing a whole set of new functions.
      79b5ccbf
    • Ondřej Surý's avatar
      Refactor isc_timer_reset() use with semantic patch · e6ca2a65
      Ondřej Surý authored and Evan Hunt's avatar Evan Hunt committed
      Add and apply semantic patch to remove expires argument from the
      isc_timer_reset() calls through the codebase.
      e6ca2a65
    • Ondřej Surý's avatar
      Remove expires argument from isc_timer API · 6437bcc4
      Ondřej Surý authored and Evan Hunt's avatar Evan Hunt committed
      The isc_timer_reset() now works only with intervals for once timers.
      
      This makes the API almost 1:1 compatible with the libuv timers making
      the further refactoring possible.
      6437bcc4
    • Ondřej Surý's avatar
      Change isc_timer_reset() usage to never use expires argument · 27850a5a
      Ondřej Surý authored and Evan Hunt's avatar Evan Hunt committed
      There were two places where expires argument (absolute isc_time_t value)
      was being used.  Both places has been converted to use relative interval
      argument in preparation of simplification and refactoring of isc_timer
      API.
      27850a5a
    • Ondřej Surý's avatar
      Refactor isc_timer_create() to just create timer · c259cecc
      Ondřej Surý authored and Evan Hunt's avatar Evan Hunt committed
      The isc_timer_create() function was a bit conflated.  It could have been
      used to create a timer and start it at the same time.  As there was a
      single place where this was done before (see the previous commit for
      nta.c), this was cleaned up and the isc_timer_create() function was
      changed to only create new timer.
      c259cecc
    • Ondřej Surý's avatar
      Change lib/dns/nta.c to create inactive timer and then reset it · 514053f2
      Ondřej Surý authored and Evan Hunt's avatar Evan Hunt committed
      In nta.c, it was the only place where the active timer was created
      directly instead of first creating inactive timer and then starting it
      with isc_timer_reset().
      
      Change the code to create inactive timer first, so we can refactor the
      isc_timer_create() function.
      514053f2
    • Ondřej Surý's avatar
      Remove "a temporary hack, 'rndc timerpoke'" · 8fbb42c4
      Ondřej Surý authored and Evan Hunt's avatar Evan Hunt committed
      In 2002, "a temporary hack, 'rndc timerpoke'" was added.  It's time
      for it to go, so it was removed.
      8fbb42c4
    • Ondřej Surý's avatar
      Remove unused isc_timer_touch() function · f4751a91
      Ondřej Surý authored and Evan Hunt's avatar Evan Hunt committed
      The isc_timer_touch() was unused, just remove it.
      f4751a91
    • Ondřej Surý's avatar
      Remove isc_timertype_limited from isc_timer API · bbe1c06a
      Ondřej Surý authored and Evan Hunt's avatar Evan Hunt committed
      The isc_timertype_limited timer type was never used (not even in tests).
      Remove isc_timertype_limited timer type before planned refactoring.
      bbe1c06a
    • Petr Špaček's avatar
      Merge branch 'pspacek/manpage-hyperlinks-fix' into 'main' · c752dff3
      Petr Špaček authored
      Fix dig option hyperlinks in the TSIG section of the ARM
      
      See merge request !5979
      c752dff3
    • Petr Špaček's avatar
      Fix dig option hyperlinks in the TSIG section of the ARM · f98a6a53
      Petr Špaček authored
      While backporting !5934 I noticed a copy&paste mistake in TSIG
      chapter of the ARM.
      
      The incorrect reference was introduced by "Add hyperlinks from
      program options to definition in man pages" commit but it is not
      worth creating separate MR for that when the backport is not merged
      yet.
      
      (cherry picked from commit 4daef4a2)
      f98a6a53
    • Petr Špaček's avatar
      Merge branch 'pspacek/manpage-hyperlinks' into 'main' · 49d2a12e
      Petr Špaček authored
      Add hyperlinks to manual pages
      
      See merge request !5934
      49d2a12e
    • Tony Finch's avatar
      Regenerate the named.conf manual with hyperlinks · ad5b0402
      Tony Finch authored and Petr Špaček's avatar Petr Špaček committed
      The named.conf grammar is exported to the manual via
      doc/misc/rst-options.pl which is the ultimate source
      for the non-grammar parts of the man page.
      ad5b0402
    • Petr Špaček's avatar
      Add internal hyperlinks to See Also section of manual pages · 1d4d008f
      Petr Špaček authored
      Replace :manpage: with :iscman: to generate internal hyperlinks. That
      way reader can use links even when offline, and jumps to man pages
      for the same version.
      
      Formerly HTML version of man pages did not have links in See Also
      section because :manpage: role in Sphinx can generate only external
      hyperlinks - and we do not have that enabled.
      Enabling the Sphinx :manpage: linking could reliably create hyperlinks
      only to external URLs, but that would take users to another version
      of docs.
      
      Generated by:
          find bin -name '*.rst' | xargs sed -i -e 's/:manpage:`\([^(]\+\)(\([0-9]\))`/:iscman:`\1(\2) <\1>`/g'
      + hand-edit to revert change for mmencode reference which is
        not provided in our source tree.
      1d4d008f
    • Petr Špaček's avatar
      420a71df
    • Petr Špaček's avatar
      Hyperlink program names to their manual pages · 53a57760
      Petr Špaček authored
      Use the new role :iscman: to replace all occurences or ``binary``
      with :iscman:`binary`, creating a hyperlink to the manual page.
      
      Generated using:
          find bin -name *.rst | xargs fgrep --files-with-matches '.. iscman' | xargs -I{} -n1 basename {} .rst > /tmp/progs
          for PROG in $(cat /tmp/progs); do find -name '*.rst' | xargs sed -i -e "s/\`\`$PROG\`\`/:iscman:\`$PROG\`/g"; done
      
      Additional hand-edits were done mainly around filter-aaaa and
      filter-a which are program names and and option names at the
      same time. Couple more edits was neede to fix .rst syntax broken by
      automatic replacement.
      53a57760
    • Petr Špaček's avatar
      Use semantic markup for :program: self-references · c7085be2
      Petr Špaček authored
      Sphinx has it's own :program: syntax for refering to program names.
      Use it for self-references in manual pages. These self-references are
      not clickable and not as eye-cathing as links, which is a good thing.
      There is no point in attracting attention to ``dig`` several times on a
      single page dedicated to dig itself.
      
      Substituted automatically using:
          find bin  -name *.rst | xargs fgrep --files-with-matches '.. program' | xargs -n1 bash /tmp/repl.sh
      
      With /tmp/repl.sh being:
          BASE=$(basename "$1" .rst)
          sed -i -e "s/\`\`$BASE\`\`/:program:\`$BASE\`/g" "$1"
      c7085be2
    • Petr Špaček's avatar
      Introduce new Sphinx role iscman for ISC manual pages · 7e7a946d
      Petr Špaček authored
      The new directive and role "iscman" allow to tag & reference man pages in
      our source tree. Essentially it is just namespacing for ISC man pages,
      but it comes with couple benefits.
      
      Differences from .. _man_program label we formerly used:
      - Does not expand :ref:`man_program` into full text of the page header.
      - Generates index entry with category "manual page".
      - Rendering style is closer to ubiquitous to the one produced
        by ``named`` syntax.
      
      Differences from Sphinx built-in :manpage: role:
      - Supports all builders with support for cross-references.
      - Generates internal links (unlike :manpage: which generates external
        URLs).
      - Checks that target exists withing our source tree.
      7e7a946d
    • Tony Finch's avatar
      More man page option hyperlinks · ccc63783
      Tony Finch authored and Petr Špaček's avatar Petr Špaček committed
      The dig man page wanted -h option hyperlink and anchor, and there
      were a couple of missing cross-references in the rndc man page.
      ccc63783
    • Petr Špaček's avatar
      Add hyperlinks from program options to definition in man pages · a85df3ff
      Petr Špaček authored
      Side-effect of hyperlinking is that typos in program and option names
      are now detected by Sphinx.
      
      Candidate -options were detected using:
          find -name *.rst | xargs grep '``-[^`]'
      and then modified from ``-o`` to :option:`-o` using regex
          s/``\(-[^`]\+\)``/:option:`\1`/
      + manual modifications where necessary.
      
      Non-hyphenated options were detected by looking at context around
      program names:
          find bin -name *.rst | xargs -I{} -n1 basename {} .rst | sort -u
      and grepping for program name with trailing whitespace.
      
      Stand-alone program names like ``named`` are not hyperlinked in this
      commit.
      a85df3ff
    • Petr Špaček's avatar
      Fix rndc command in release notes for 9.17.12 · 5f0ee7c3
      Petr Špaček authored
      rndc checkds does not exist, it should have been rndc dnssec
      
      Related: #2488, !4813
      5f0ee7c3
    • Petr Špaček's avatar
      Add semantic markup for program names into manual pages · 8537878c
      Petr Špaček authored
      It allows to cross-reference options in man pages from other
      documents using :option:`named -g` syntax.
      8537878c
    • Petr Špaček's avatar
      Denote all command line options using semantic markup (.. option::) · ec30944a
      Petr Špaček authored
      The markup allows referencing individual options, and also makes them
      more legible (no more thin red text on gray background).
      
      Most of the work was done using regexes:
          s/^``-\(.*\)``$/.. option:: -\1\r/
          s/^``+\(.*\)``$/.. option:: +\1\r/
      on bin/**/*.rst files along with visual inspection and hand-edits,
      mostly for positional arguments.
      
      Regex for rndc.rst:
          s/^``\(.*\)``/.. option:: \1\r/
      + hand edits to remove extra asterisk and whitespace here and there.
      ec30944a
    • Michał Kępień's avatar
      Merge branch 'michal/tidy-setup-of-python-based-tests' into 'main' · e9f4d00b
      Michał Kępień authored
      Tidy setup of Python-based tests
      
      See merge request !5960
      e9f4d00b
    • Michał Kępień's avatar
      Tweak Automake conditionals for pytest-based tests · 173ad9cf
      Michał Kępień authored
      Since pytest itself skips tests using dnspython if the latter is not
      available, also using Automake conditionals for silently skipping
      pytest-based tests requiring dnspython is redundant and hides
      information.  Allow all pytest-based tests requiring dnspython to be run
      whenever pytest itself is available, in order to ensure test skipping is
      done in a uniform manner.
      
      Note that the above reasoning only applies to pytest-based tests, so
      similar adjustments were not made for shell-based tests using Python
      scripts that require dnspython ("chain", "cookie", "dnssec", "qmin").
      173ad9cf
    • Michał Kępień's avatar
      Rework skipping long tests · 00392921
      Michał Kępień authored
      The ability to conveniently mark tests which should only be run when the
      CI_ENABLE_ALL_TESTS environment variable is set seems to be useful on a
      general level and therefore it should not be limited to the "timeouts"
      system test, where it is currently used.
      
      pytest documentation [1] suggests to reuse commonly used test markers by
      putting them all in a single Python module which then has to be imported
      by test files that want to use the markers defined therein.  Follow that
      advice by creating a new bin/tests/system/pytest_custom_markers.py
      Python module containing the relevant marker definitions.
      
      Note that "import pytest_custom_markers" works from a test-specific
      subdirectory because pytest modifies sys.path so that it contains the
      paths to all parent directories containing a conftest.py file (and
      bin/tests/system/ is one).  PyLint does not like that, though, so add a
      relevant PyLint suppression.
      
      The above changes make bin/tests/system/timeouts/conftest.py redundant,
      so remove it.
      
      [1] https://docs.pytest.org/en/7.0.x/how-to/skipping.html#id1
      00392921
    • Michał Kępień's avatar
      Rework imports in dnspython-based system tests · 49312d6b
      Michał Kępień authored
      Ensure all "import dns.*" statements are always placed after
      pytest.importorskip('dns') calls, in order to allow the latter to
      fulfill their purpose.  Explicitly import all dnspython modules used by
      each dnspython-based test to avoid relying on nested imports.  Replace
      function-scoped imports with global imports to reduce code duplication.
      49312d6b
    • Michał Kępień's avatar
      Fix skipping tests requiring dnspython · 05c97f23
      Michał Kępień authored
      The intended purpose of the @pytest.mark.dnspython{,2} decorators was to
      cause dnspython-based tests to be skipped if dnspython is not available
      (or not recent enough).  However, a number of system tests employing
      those decorators contain global "import dns.resolver" statements which
      trigger ImportError exceptions during test initialization if dnspython
      is not available.  In other words, the @pytest.mark.dnspython{,2}
      decorators serve no useful purpose.
      
      Currently, whenever a Python-based test requires dnspython, that
      requirement applies to all tests in a given *.py file.  Given that,
      employ global pytest.importorskip() calls to ensure dnspython-based
      parts of various system tests are skipped when dnspython is not
      available.  Remove all occurrences of the @pytest.mark.dnspython{,2}
      decorators (and all associated code) to prevent confusion.
      05c97f23
    • Michał Kępień's avatar
      Fix skipping tests requiring the requests module · 704ad290
      Michał Kępień authored
      The intended purpose of the @pytest.mark.requests decorator was to cause
      Python-based parts of the "statschannel" system test to be skipped if
      the requests Python module is not available.  However, both
      tests-json.py and tests-xml.py contain a global "import requests"
      statement which triggers ImportError exceptions during test
      initialization if the requests module is not available.  In other words,
      the @pytest.mark.requests decorator serves no useful purpose.
      
      Since all tests in both tests-json.py and tests-xml.py depend on the
      requests Python module, employ pytest.importorskip() to ensure the
      Python-based parts of the "statschannel" system test are skipped when
      the requests module is not available.  Remove all occurrences of the
      @pytest.mark.requests decorator (and all associated code) to prevent
      confusion.
      704ad290
    • Michał Kępień's avatar
      Simplify skipping tests depending on libxml2 · 286b57c7
      Michał Kępień authored
      All tests in bin/tests/system/statschannel/tests-xml.py require libxml2
      support to be enabled in BIND 9 at build-time.  Instead of applying the
      same pytest.mark.skipif() decorator to every test in that file, set the
      'pytestmark' global accordingly in order to immediately skip all tests
      in tests-xml.py if libxml2 support is not compiled in.
      
      Remove all occurrences of the @pytest.mark.xml decorator (and all
      associated code) from the "statschannel" system test as the
      xml.etree.ElementTree module is a part of the Python standard library
      since Python 2.5 (so checking whether it is available is redundant) and
      checking for libxml2 support in the tested BIND 9 build is already
      handled by setting the 'pytestmark' global accordingly.
      286b57c7
    • Michał Kępień's avatar
      Simplify skipping tests depending on json-c · 0a76f186
      Michał Kępień authored
      All tests in bin/tests/system/statschannel/tests-json.py require json-c
      support to be enabled in BIND 9 at build-time.  Instead of applying the
      same pytest.mark.skipif() decorator to every test in that file, set the
      'pytestmark' global accordingly in order to immediately skip all tests
      in tests-json.py if json-c support is not compiled in.
      
      Remove all occurrences of the @pytest.mark.json decorator (and all
      associated code) from the "statschannel" system test as the json module
      is a part of the Python standard library since Python 2.6 (so checking
      whether it is available is redundant) and checking for json-c support in
      the tested BIND 9 build is already handled by setting the 'pytestmark'
      global accordingly.
      
      Also remove a related excerpt from bin/tests/system/rpzextra/conftest.py
      as it is a copy-paste artifact that serves no purpose in the "rpzextra"
      system test.
      0a76f186
    • Michał Kępień's avatar
      Refactor "statschannel" test's helper modules · 96b7f9f9
      Michał Kępień authored
      The "statschannel" system test contains two Python helper modules:
      
        - generic.py: test functions directly invoked by both tests-json.py
          and test-xml.py,
      
        - helper.py: helper functions invoked by test functions in generic.py.
      
      The above logic for splitting helper functions into Python modules
      prevents selective test skipping from working due to unconditional
      import statements being present in both helper modules.  For example, if
      dnspython is not available on the test host, tests-json.py imports
      generic.py, which in turn imports helper.py, which in turn attempts to
      import various dnspython modules, triggering ImportError exceptions
      during test initialization.  Various decorators used for some tests
      (like @pytest.mark.dnspython) suggest that such a scenario should be
      handled gracefully, but that is not the case - modifying the test
      collection in conftest.py does not prevent pytest from failing due to
      import errors.
      
      Fix by moving helper functions around to achieve a different split:
      
        - generic.py: helper functions only relying on the Python standard
          library,
      
        - generic_dnspython.py: helper functions requiring dnspython.
      
      Only two tests in tests-{json,xml}.py need dnspython to work
      (test_traffic_json(), test_traffic_xml()).  Since all
      dnspython-dependent code is now present in generic_dnspython.py, employ
      pytest.importorskip() in those two tests to ensure they can be
      selectively skipped when dnspython is not available.  Adjust other code
      to account for the revised Python helper module layout.  Remove all
      occurrences of the @pytest.mark.dnspython decorator (and all associated
      code) from the "statschannel" system test to prevent confusion.
      96b7f9f9
    • Michał Kępień's avatar
      Improve test discovery logic in get_ports.sh · 4e0d5768
      Michał Kępień authored
      The find invocation used by the bin/tests/system/get_ports.sh script
      ("find . -maxdepth 1 -mindepth 1 -type d") assumes the list of
      directories in bin/tests/system/ remains unchanged throughout the run
      time of a single system test suite.  With pytest in use and the
      conftest.py file now present in bin/tests/system/, that assumption is no
      longer true as a __pycache__ directory may be created when the first
      pytest-based test is started.  Since the list of names returned by the
      above find invocation serves as a fixed-size array of "port range
      slots", any changes to that list during a system test suite run may lead
      to port assignment collisions [1].
      
      Fix by making the find invocation more nuanced, so that it only returns
      names of directories containing test code.  Squash a grep / cut pipeline
      into a single awk invocation.
      
      [1] see commit 31e5ca4b
      4e0d5768
    • Michał Kępień's avatar
      Reuse common port-related test fixtures · 53ef8835
      Michał Kępień authored
      Most Python-based system tests need to know which ports were assigned to
      a given test by bin/tests/system/get_ports.sh.  This is currently
      handled by inspecting the values of various environment variables (set
      by bin/tests/system/run.sh) and passing the port numbers to Python
      scripts via pytest fixtures.  However, this glue code has so far been
      copy-pasted into each system test using it, rather than reused.
      
      Since pytest also looks for conftest.py files in parent directories,
      move commonly used fixtures to bin/tests/system/conftest.py.  Set the
      scope of all the moved fixtures to "session" as their return values are
      only based on environment variables, so there is no point in recreating
      them for every test requesting them.  Adjust test code accordingly.
      53ef8835
  4. 13 Mar, 2022 1 commit