dev.md 60.3 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
<!--
Copyright (C) Internet Systems Consortium, Inc. ("ISC")

SPDX-License-Identifier: MPL-2.0

This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0.  If a copy of the MPL was not distributed with this
file, you can obtain one at https://mozilla.org/MPL/2.0/.

See the COPYRIGHT file distributed with this work for additional
information regarding copyright ownership.
-->

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
## BIND Developer Information

### Contents

1. [The code review process](#reviews)
1. [Testing](#testing)
    * [System tests](#systest)
    * [Unit tests](#unittest)
1. [BIND system architecture](#arch)
    * [Source tree layout](#layout)
    * [Design by contract](#dbc)
    * [Magic numbers](#magic)
    * [Result codes](#results)
    * [Memory management](#mem)
    * [Lists](#lists)
    * [Buffers and regions](#buffers)
    * [Names](#names)
31
    * [Rdata Classes](#rdata)
32
33
34
35
36
    * [Iterators](#iterators)
    * [Logging](#logging)
    * [Adding a new RR type](#rrtype)
    * [Task and timer model](#tasks)

Evan Hunt's avatar
Evan Hunt committed
37
### <a name="reviews"></a>The code review process
38

39
Every line of code committed to BIND has been reviewed by ISC engineers
40
41
42
43
44
45
46
47
48
49
first.

The code review process is a dialog between the original author and the
reviewer.  Code inspection, including documentation and tests, is part of
this.  Compiling and running the resulting code should be done in most
cases, even for trivial changes, to ensure that it works as intended. In
particular, a full regression test (`make` `check`) must be run for every
modification so that unexpected side-effects are identified.

When a problem or concern is found by the reviewer, these comments are
50
placed on the merge request in GitLab so the author can respond.
51
52
53
54
55
56
57
58
59

#### What is reviewed:

First, consideration is given to whether contributed code would
be useful to a significant user base (we can't take on the additional
maintenance and support burden for changes that would only be useful
to a tiny niche).  Second, whether the approach taken is consistent
with ISC's open-internet goals, BIND architecture, and DNS best
practices.  Third, the contribution is checked for correctness and
60
completeness.
61
62

Obvious bottlenecks and places where performance or reliability may suffer
63
64
65
66
67
68
69
70
are noted as part of the review.

New functions must be adequately commented. Public API functions are
documented in the corresponding header file, static functions in the C
file, above the function header.  Particularly complex code should be
commented throughout the function body as well.

A patch is much more likely to be accepted quickly if it includes
71
72
73
tests providing good coverage of the new code.  Tests for bugfix code
should fail when run against the unmodified code; tests for new feature
code should have good code coverage and address corner cases and error
74
75
cases.  Newly added API functions should have unit tests if possible.
(See [testing](#testing).)
76
77
78
79
80
81
82
83
84
85

Documentation is also reviewed. This includes all user-facing text,
including log messages, manual pages, user manuals and sometimes even
comments; they must be clearly written and consistent with existing style.

#### Steps in code review:

* Read the diff
* Read accompanying notes in the ticket
* Apply the diff to the appropriate branch
86
* Run `configure` (using at least `--enable-developer`)
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
* Build
* Read the documentation, if any
* Read the tests
* Run the tests
  <br>(In some cases it may be appropriate to run tests against code
  from before the change to ensure that they fail as expected.)

#### Things we look for

* General correctness of approach
* Style errors
* Simple coding errors
* Files inadvertently omitted
* Unnecessarily complex code
* Complex code with insufficient comments
* Lack of boundary checking
* Memory and resource leaks (deallocations must match allocations)
* Places that need `REQUIRE` or `INSIST`
* Thread safety
* Bad function names/variable names
* Overly long functions
* Copies of code that could be unified in a helper function
* Premature optimizations
* Compiler warnings introduced
111
* Portability issues, such as the use of non-POSIX library calls or options
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
* DNS/protocol problems
* Cut/pasted code that may have been modified in one place but needs to be modified in other places as well
* No tests or inadequate tests
* Testability problems
* No documentation or inadequate documentation
* Grammar, spelling and clarity problems in documentation
* Usability problems

When a patch is contributed which is a good idea but doesn't meet our code
quality requirements, we will often keep the ticket open so that we can
address the issue ourselves later.

Sometimes contributed code is fine, but ISC staff still have to add
documentation and/or tests -- that's okay, but it may take a long time to
get to the top of our priority list.  Ensuring that your patch includes
tests and documentation will reduce delay.

### <a name="testing"></a> Testing

#### <a name="systest"></a> Running system tests

To enable system tests to work, we first need to create the test loopback
interfaces (as root):

        $ cd bin/tests/system
        $ sudo sh ifconfig.sh up
        $ cd ../../..

140
To run the tests, build BIND (be sure to use --with-cmocka to run unit
141
142
143
tests), then run `make` `check`.  An easy way to check the results:

        $ make check 2>&1 | tee /tmp/check.out
144
        $ grep -A 10 'Testsuite summary' /tmp/check.out
145
146
147
148
149
150
151
152
153

This will show all of the test results. One or two "R:SKIPPED" is okay; if
there are a lot of them, then you probably forgot to create the loopback
interfaces in the previous step. (NOTE: the summary of tests that appears at
the end of `make` `check` only summarizes the system test results, not the
unit tests, so you can't rely on it to catch everything.)

To run only the system tests, omitting unit tests:

154
155
	$ make test

Matthijs Mekking's avatar
Matthijs Mekking committed
156
To run an individual system test:
157

Matthijs Mekking's avatar
Matthijs Mekking committed
158
159
160
161
162
        $ make -C bin/tests/system/ check TESTS=<testname> V=1

Or:

        $ TESTS= make -e all check
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
        $ cd bin/tests/system
        $ sh run.sh <testname>

System tests are in separate directories under `bin/tests/system`.
For example, the "dnssec" test is in `bin/tests/system/dnssec`.

#### Writing system tests

The following standard files are found in system test directories:

- `prereq.sh`: run at the beginning to determine whether the test can be run at all; if not, we see R:SKIPPED

- `setup.sh`: sets up the preconditions for the tests

- `tests.sh`: runs all the test cases. A non-zero return value results in R:FAIL

- `clean.sh`: run at the end to clean up temporary files, but only if the
  test was completed successfully; otherwise the temporary files are left
  in place for inspection.

- `ns[X]`: these subdirectories contain test name servers that can be
  queried or can interact with each other. (For example, `ns1` might be
  running as a root server, `ns2` as a TLD server, and `ns3` as a recursive
  resolver.)  The value of X indicates the address the server listens on:
  for example, `ns2` listens on 10.53.0.2, and ns4 on 10.53.0.4. All test
  servers use port 5300 so they don't need to run as root. All servers
  log at the highest debug level, and the logs are captured in the file
  `nsX/named.run`.

- `ans[X]`: like `ns[X]`, but these are simple mock name servers
  implemented in perl; they are generally programmed to misbehave in ways
  `named` wouldn't, so as to exercise `named`'s ability to interoperate with
  badly behaved name servers.  Logs, if any, are captured in `ansX/ans.run`.

All test scripts source the file `bin/tests/system/conf.sh` (which is
generated by `configure` from `conf.sh.in`).  This script provides
functions and variables pointing to the binaries under test; for example,
`DIG` contains the path to `dig` in the build tree being tested, `RNDC`
points to `rndc`, `SIGNZONE` to `dnssec-signzone`, etc.

#### <a name="unittest"></a> Building unit tests

205
BIND uses the cmocka unit testing framework.
206

207
208
To build BIND with unit tests, run `configure` with the `--with-cmocka`
option.  This requires cmocka >= 1.0.0 to be installed in the system.
209
210
211
212
213
214
215
216
217
218
219
220

#### Running unit tests

Unit tests are stored in `/tests` subdirectories under the libraries
they test.  For example, the unit tests for libisc are in `lib/isc/tests`.
Particular test sets are called `{module}_test.c`, where {module} is
usually the name of the module being tested; `rbt_test.c` tests functions
in `rbt.c`.  (There are exceptions to this rule, though; for instance,
`hash_test.c` tests hash functions that are implemented in several
different files in `lib/isc`.)

When BIND is built with unit tests, they will be run as part of
221
`make` `check`.  But if you want to run *only* the unit tests:
222

223
        $ make unit
224
225
226
227
228
229
230
231
232
233
234
235

You can also run the unit tests for only one library:

        $ cd lib/isc/tests (or lib/dns/tests)
        $ make unit

Or run a particular test case (in the following example, the isc_sha512
test case in the hash unit test).  This has the advantage that you can see
whatever output the unit test emits, whereas in the other modes, output is
redirected:

        $ cd lib/isc/tests
236
        $ ./hash_test isc_sha512
237
238
239

#### Writing unit tests

240
241
242
243
244
Information on writing cmocka tests can be found at the
[cmocka website](https://cmocka.org).

New unit tests should be added whenever new API functionality is added to the
libraries.
245
246
247
248
249
250

### <a name="arch"></a> BIND system architecture

#### <a name="layout"></a> Source tree layout

* `bind9/bin`: binaries
251
    * `bind9/bin/named`: source code for the `named` binary; includes server configuration, interface manager, client manager, and high-level processing logic for query, update, and xfer.
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
    * `bind9/bin/dnssec`: DNSSEC-related tools written in C:
      `dnssec-keygen`, `dnssec-signzone`, `dnssec-settime`,
      `dnssec-revoke`, `dnssec-keyfromlabel`, `dnssec-dsfromkey`,
      `dnssec-verify` (BIND 9.9+)
    * `bind9/bin/rndc`: `rndc` binary
    * `bind9/bin/dig`: `dig`, `host`, and `nslookup`
    * `bind9/bin/delv`: `delv`
    * `bind9/bin/check`: `named-checkconf` and `named-checkzone`
    * `bind9/bin/confgen`: `rndc-confgen`, `ddns-confgen`, and
      `tsig-keygen` (BIND 9.9+)
    * `bind9/bin/tools`: assorted useful tools: `named-journalprint`,
      `nsec3hash`, etc
* `bind9/lib`: libraries
    * `bind9/lib/isc`: implements basic functionality such as threads,
      tasks, timers, sockets, memory manager, buffers, and basic data types.
        * `bind9/lib/isc/tests`: unit tests for libisc
    * `bind9/lib/dns`: implements higher-level DNS functionality:
      red-black trees, rdatasets, views, zones, ACLs, resolver, validator, etc
        * `bind9/lib/dns/tests`: unit tests for libdns
    * `bind9/lib/bind9`: library implementing bind9-specific functionality,
      principally configuration validity checking (used in `named` and
      `named-checkconf` when reading `named.conf`).
    * `bind9/lib/isccfg`: library implementing the `named.conf`
      configuration parser.
    * `bind9/lib/isccc`: library implementing the control channel used
      by `rndc`
    * `bind9/lib/irs`: provides mechanisms for reading `/etc/resolv.conf`
279
      and other configuration files.
280
281
282

#### Namespace

Evan Hunt's avatar
Evan Hunt committed
283
284
See the [namespace](style.md#public_namespace) discussion in the
[BIND coding style](style.md) document.
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379

#### <a name="dbc"></a>Design by contract

BIND uses the "Design by Contract" pattern for most function calls.

A quick summary of the idea is that a function and its caller make a
contract.  If the caller meets certain preconditions, then the function
promises to either fulfill its contract (i.e. guarantee a set of
postconditions), or to clearly fail.

"Clearly fail" means that if the function cannot succeed, then it will
not silently fail and return a value which the caller might interpret as
success.

If a caller doesn't meet the preconditions, then "further execution is
undefined".  The function can crash, compute a garbage result, fail
silently, etc.  Allowing the function to define preconditions greatly
simplifies many APIs, because the API need not specify a way of saying
"hey caller, the values you passed in are garbage".

Typically, preconditions are specified in the functions .h file, and
encoded in its body with `REQUIRE` statements.  The `REQUIRE` statements
cause the program to dump core if they are not true, and can be used to
identify callers that are not meeting their preconditions.

Postconditions can be encoded with `ENSURE` statements.  Within the body of
a function, `INSIST` is used to assert that a particular expression must be
true.

Assertions must not have side effects that the function relies upon,
because assertion checking may be turned off in some environments.
(This is *not* recommended, however: assertion failures serve the
useful function of ensuring that `named` does not continue running
in an insane state.  The surfeit of assertions in BIND 9 have made
it vulnerable over the years to "packets of death" and other
denial-of-service exploits, but as of this writing - more than 14
years since the initial release - BIND 9 has never had an arbitrary
code execution vulnerability.)

#### <a name="magic"></a>Magic numbers

A number of data structures in the ISC and DNS libraries have an
`unsigned int magic` value as the first field.  The purpose of the
magic number is principally to validate that a pointer that's been
passed to a subroutine really points to the type it claims to be.  This
helps detect problems caused by resources being freed prematurely, that
have been corrupted, or that have not been properly initialized.  It can
also be handy in debugging.

Magic numbers should always be the first field in a structure.  They never
require locking to access.  As to the actual value to be used, something
mnemonic is good:

        #define TASK_MAGIC                      0x5441534BU     /* TASK. */
        #define VALID_TASK(t)                   ((t) != NULL && \
                                                 (t)->magic == TASK_MAGIC)

        #define TASK_MANAGER_MAGIC              0x54534B4DU     /* TSKM. */
        #define VALID_MANAGER(m)                ((m) != NULL && \
                                                 (m)->magic ==
                                                  TASK_MANAGER_MAGIC)

Unless the memory cost is critical, most objects should have a magic number.

The magic number should be the last field set in a creation routine, so that
an object will never be stamped with a magic number until it is valid.

The magic number should be set to zero immediately before the object is
freed.

Magic values are usually private to the implementation of the type;
i.e.  they are defined in the .c file, not the .h file.  There are some
exceptions to this.

Validation of magic numbers is done by routines that manipulate the type,
not by users of the type.  (Indeed, user validation is usually not possible
because the magic number is not public.)

#### <a name="results"></a>Result codes

The `isc_result_t` type is provided for function result codes,
and is used throughout BIND.  For example:

        isc_result_t result;
        FILE *fp = NULL;

        result = isc_stdio_open("file", "r", &fp);

Note that an explicit result code is used, instead of mixing the error
result type with the normal result type.  In contrast to the
C library routine `fopen()` which returns a file pointer or `NULL`
on failure (setting `errno` to indicate what the nature of the problem
was), BIND style always keeps indication of the function's success or
failure separate from its returned data.  Similarly, the C library
function `fread()` returns the number of characters read and then
380
depends on `feof()` and `ferror()` to determine whether an error occurred
381
382
383
384
385
or the end of file was reached, but BIND's version uses result codes:

        char buffer[BUFSIZ];
        size_t n;

386
        result = isc_stdio_read(buffer, 1, sizeof(buffer), fp, &n);
387
388
389
390
391
392
393
394
395
396
397
        if (result == ISC_R_SUCCESS) {
                /* Do something with 'buffer'. */
        } else if (result == ISC_R_EOF) {
                /* EOF. */
                result = ISC_R_SUCCESS;
        } else {
                /* Some other error occurred. */
        }

Only functions which cannot fail (assuming the caller has provided valid
arguments) should return data directly instead of a result code.  For
398
example, `dns_name_issubdomain()` returns an `bool`, because it
399
400
401
402
403
404
has no failure mode.

A result code can be converted to a human-readable error message by
calling `isc_result_totext(result)`.

Many result codes have been defined and can be found in the source tree
405
in `lib/isc/include/isc/result.h`.
406
407

ISC library result codes (many of which are generically useful elsewhere)
408
begin with `ISC_R`: examples include `ISC_R_SUCCESS`, `ISC_R_FAILURE`,
409
`ISC_R_NOMEMORY`, etc.
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467

DNS library result codes begin with `DNS_R`: `DNS_R_SERVFAIL`, `DNS_R_NXRRSET`,
etc).  Other sets of result codes are defined for crypto functions (`DST_R`
and `PKCS_R`).

For portability, ISC result codes are used instead of codes provided
by the operating system; for example, `ISC_R_NOMEMORY` instead of
`ENOMEM`.  In some cases, but not all, POSIX-defined error codes can be
converted to an ISC result code by calling `isc__errno2result(errno)`.
This can't be relied on; there are too many OS-specific error codes to
provide meaningful translations for all of them.  Unknown `errno` values
are converted to `ISC_R_UNEXPECTED`.

#### <a name="buffers">Buffers and regions

A useful set of functions is provided for manipulating memory
buffers: the `isc_buffer` API.  Buffers can be used for parsing
and constructing messages in both text and binary formats.

A buffer is associated with a region of memory, which is subdivided
into 'used' and 'available'.  The 'used' subregion is further subdivided
into 'consumed' and 'remaining'.

When parsing a message, the message to be parsed in in the 'used'
part of the buffer.  As the message is parsed, the 'consumed'
subregion grows and the 'remaining' subregion shrinks. 

When creating a message, data is written into the 'available'
subregion, which then becomes part of 'used'.

The current sizes of these subregions can be determined by calling
`isc_buffer_usedlength()`, `isc_buffer_consumedlength()`,
`isc_buffer_remaininglength()`, and `isc_buffer_availablelength()`.

The memory associated with a buffer may be dynamically allocated
from a memory context using `isc_buffer_allocate()` and freed by
`isc_buffer_free()`, or it may be a static region of memory
with which we want to use buffer semantics.  In that case, we
associate a new buffer object with the desired block of memory
by running `isc_buffer_init()`.  If the intention is to write
to the memory, nothing further is necessary; if it is to read
the memory using buffer sementaics, then we must mark the memory
as part of the 'used' subregion:

        isc_buffer_t b;
        char text[BUFSIZ];
        unsigned int n;

        result = isc_stdio_read(buf, 1, BUFSIZ, fp, &n);
        if (result == ISC_R_SUCCESS && n > 0U) {
                isc_buffer_init(&b, text, sizeof(text));
                isc_buffer_add(&b, n);
                /* now we can read the buffer */
        }

Several functions are provided for both reading and writing
to the buffer:

Evan Hunt's avatar
Evan Hunt committed
468
469
* `isc_buffer_getuint8()`: Read and return an 8-bit unsigned integer
* `isc_buffer_putuint8()`: Write an 8-bit unsigned integer to a buffer
470

Evan Hunt's avatar
Evan Hunt committed
471
* `isc_buffer_getuint16()`: Read a 16-bit unsigned integer in
472
  network byte order, convert to host byte order, and return it
Evan Hunt's avatar
Evan Hunt committed
473
* `isc_buffer_putuint16()`: Convert an unsigned 16-bit integer from
474
475
  host to network byte order and write it to a buffer.

Evan Hunt's avatar
Evan Hunt committed
476
* `isc_buffer_getuint32()`: Read a 32-bit unsigned integer in
477
  network byte order, convert to host byte order, and return it
Evan Hunt's avatar
Evan Hunt committed
478
* `isc_buffer_putuint32()`: Convert an unsigned 32-bit integer from
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
  host to network byte order and write it to a buffer.

* `isc_buffer_putstr()`: Copy a null-terminated string into a buffer
* `isc_buffer_putmem()`: Copy a fixed-length region of memory into a
  buffer.

A simpler set of functions have also been provided for handling
memory regions: the `isc_region` API.  A region is a simple structure
that only contains a base pointer (to the beginning of the associated
memory) and a length.  Buffers and buffer subregions can be converted to
regions using `isc_buffer_region()`, `isc_buffer_usedregion()`, etc.
Regions can be copied to buffers by using `isc_buffer_copyregion()`,
or simply by running `isc_buffer_init()` on the region's base pointer.

#### <a name="mem"></a>Memory management

495
BIND tracks its memory usage internally via "memory contexts".  Multiple
496
497
separate memory contexts can be created for the use of different modules or
subcomponents, and each can have its own size limits and tuning parameters
498
499
500
and maintain its own statistics, allocations and free lists.  Memory
allocation is based on the `jemalloc` library on platforms where the library
is available.
501
502
503
504
505
506
507
508
509
510

The memory system helps with diagnosis of common coding errors such as
memory leaks and use after free. Newly allocated memory is populated with
the repeating value 0xbe, and freed memory with 0xde.  BIND tracks every
memory allocation, and will complain (via an assertion failure) if any
memory has not been freed when BIND shuts down.

To create a basic memory context, use:

        isc_mem_t *mctx = NULL;
511
        isc_mem_create(&mctx);
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576

When holding a persistent reference to a memory context it is advisable to
increment its reference counter using `isc_mem_attach()`.  Do not just
copy an `mctx` pointer; this may lead to a shutdown race in which the
memory context is freed before all references have been cleaned up.

        /*
         * Function to create an 'isc_foo' object.
         */
        isc_result_t
        isc_foo_create(isc_mem_t *mctx, isc_foo_t **foop) {
                isc_foo_t *foo;

                REQUIRE(mctx != NULL);
                REQUIRE(foop != NULL && *foop == NULL);

                foo = isc_mem_get(mctx, sizeof(isc_foo_t))

                /* Attach to memory context */
                isc_mem_attach(mctx, &foo->mctx);

                /* Populate other isc_foo members here */

                foo->magic = ISC_FOO_MAGIC;
                
                *foop = foo;
                return (ISC_R_SUCCESS);
        }

When finished with a memory context, detach it with `isc_mem_detach()`.
If freeing an object that contains a reference to a memory context,
you free it and detach its reference at the same time using
`isc_mem_putanddetach()`.

        void
        isc_foo_destroy(isc_foo_t **foop) {
                isc_foo_t *foo = *foop;

                /* clean up various isc_foo members */
                foo->magic = 0;

                isc_mem_putanddetach(&foo->mctx, foo, sizeof(isc_foo_t));

                *foop = NULL;
        }

Two sets of allocation and deallocation functions are provided:
`isc_mem_get()` and `isc_mem_put()`; and `isc_mem_allocate()` and
`isc_mem_free()`.

The call to `isc_mem_put()` must specify the number of bytes being freed,
so use `isc_mem_get()` when the caller can easily keep track of the size of
the allocation.

A call to `isc_mem_free()` does not need to specify the size of the
allocation, it simply frees whatever was allocated at that address, so use
`isc_mem_allocate()` when use variable size blocks of memory.

The function `isc_mem_strdup()` -- a version of `strdup()` that uses memory
contexts -- will also return memory that can be freed with
`isc_mem_free()`.

In cases where small fixed-size blocks of memory may be needed frequently,
the `isc_mempool` API can be used.  This creates a standing pool of blocks
of a specified size which can be passed out and returned without the need
577
578
579
580
581
for a new memory allocation; this can improve performance in tight inner
loops.

None of these allocation functions, including `isc_mempool_get()`, can
fail.  If no memory is available for allocation, the program will abort.
582

583
584
585
586
587
588
589
590
591
592
593
594
595
The memory context can be set to check if all memory allocated via the said
memory context was freed before the memory context was destroyed by calling
`isc_mem_checkdestroyed()`.  This could lead to false positives on abnormal
shutdowns, so the checking is only enabled in `dig` and `named` applications on
normal shutdown.

The memory context are normally used only for internal allocations, but several
external libraries allow replacing their allocators (namely libxml2, libuv and
OpenSSL).  As there has been known memory leak in the OpenSSL when
`engine_pkcs11` is loaded, memory checking at destroy is disabled by default in
the memory contexts used for external libraries and it needs to be enabled with
a `--enable-leak-detection` autoconf option.

596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
#### <a name="lists"></a>Lists

A set of macros are provided for creating, modifying and iterating
doubly-linked lists.  These are defined in `<isc/list.h>`.

To create a structure that will be part of a linked list, specify
an `ISC_LINK` as one of its members:

        typedef struct isc_foo isc_foo_t;
        struct isc_foo {
                unsigned int magic;

                /* other contents */

                ISC_LINK(isc_foo_t) link;
        };

(Note the `typedef` of `isc_foo_t` prior to the structure declaration.)

When creating an instance of this structure, initialize the link:

        isc_result_t
        isc_foo_create(isc_mem_t mctx, isc_foo_t **foop) {
                isc_foo_t *foo;

                REQUIRE(foop != NULL && *foop == NULL);

                foo = isc_mem_get(mctx, sizeof(isc_foo_t));

                ISC_LINK_INIT(foo, link);

                /* initialize other members */

                foo->magic = ISC_FOO_MAGIC;
                *foop = foo;
                return (ISC_R_SUCCESS);
        }

To make a list of these elements, first create a list variable
by declaring it using the `ISC_LIST` macro, then initialize it
with `ISC_LIST_INIT`:

        ISC_LIST(isc_foo_t) foolist;
        ISC_LIST_INIT(foolist);

The list can then be modified:

        ISC_LIST_APPEND(foolist, foo1, link);

Several macros are provided for this purpose, including `ISC_LIST_PREPEND`,
`ISC_LIST_INSERTBEFORE`, and `ISC_LIST_INSERTAFTER`.

More macros are provided for iterating the list:

        isc_foo_t *foo;
        for (foo = ISC_LIST_HEAD(foolist);
             foo != NULL; 
             foo = ISC_LIST_NEXT(foo, link))
        {
                /* do things */
        }

There are also `ISC_LIST_TAIL` and `ISC_LIST_PREV` macros for walking the
list in reverse order.

Items can be removed from the list using `ISC_LIST_UNLINK`:

        ISC_LIST_UNLINK(foolist, foo, link);

#### <a name="names"></a>Names

The `dns_name` API has facilities for processing DNS names and labels,
both dynamically and statically allocated, relative and absolute,
compressed and not, with straightforward conversions from text to
wire format and vice versa.

##### Initializing

When a name object is initialized, a pointer to an "offset table"
(`dns_offsets_t`) may optionally be supplied; this will improve
performance of most name operations if the name is used more than
once.

        dns_name_t name1, name2;
        dns_offsets_t offsets1;

        dns_name_init(&name1, &offsets1);
        dns_name_init(&name2, NULL);

##### Copying

687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
There are three methods for copying name objects:

- `dns_name_clone()` makes a target refer to the same data as the
  source, but does not copy the data. The source must not be changed
  or freed while the target is still in use.

- `dns_name_copy()` copies the source data from one name object into
  another, which must already have a dedicated buffer associated with
  it to receive the data.  The target name can have a buffer assigned
  to it using `dns_name_setbuffer()`:

        dns_name_t target;
        unsigned char namedata[DNS_NAME_MAXWIRE];
        isc_buffer_t buffer;

        isc_buffer_init(&buffer, namedata, sizeof(namedata));
        dns_name_init(&target, NULL);
        dns_name_setbuffer(target, &buffer);
        dns_name_copy(source, &target);

  Using a [fixed name](#fixedname) (see below) for the target
  ensures that it has sufficient buffer space without needing to set
  a buffer.

- `dns_name_dup()` copies a name into a new name object, dynamically
  allocating buffer space as needed. `dns_name_dupwithoffsets()` does
  the same, but also dynamically allocates space for the copied offset
  table.  Targets created by these functions must be freed by calling
  `dns_name_free()`.
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829

##### Wire format

To create a name object from a wire format message such as a DNS
query or response, use `dns_name_fromwire()`.  Generally this is
done with names in a DNS message object (`dns_message_t`), and some
names may be compressed; the ongoing decompression state for a message
is maintained in a "decompression context" object (`dns_decompress_t`)
which must be initialized before the first call to `dns_name_fromwire()`
for a given message, and passed to each additional call until all
the names have been extracted.

Similarly, `dns_name_towire()` converts name objects into DNS wire
format, using an ongoing "compression context" object (`dns_compress_t`).

##### Text format

Converting text representations of names to name objects is
usually done by calling `dns_name_fromtext()`, which converts a name
found in a source [buffer object](#buffers)

When using `dns_name_fromtext()`, the target name must have a buffer
associated with it, or else a buffer must be passed in separately which
will be used to store name data.  An `origin` parameter indicates a zone origin
name, which is appended to the converted name; for absolute names, the root
zone name, `dns_rootname`, should be used as origin.  If the
`DNS_NAME_DOWNCASE` flag is set in the `options` parameter, then the target
name will be converted to lower case, regardless of the case of the source
name.

        char *text = "foo.com";
        unsigned char namedata[DNS_NAME_MAXWIRE];
        isc_buffer_t buf;
        dns_name_t name;

        dns_name_init(&name, NULL);
        isc_buffer_init(&buf, namedata, sizeof(namedata));
        isc_buffer_add(&buf, strlen(text));
        result = dns_name_fromtext(&name, &buf, dns_rootname, 0, NULL);
        if (result != ISC_R_SUCCESS) {
                /* something went wrong */
        }

An alternate mechanism `dns_name_fromstring()` converts a standard
null-terminated string to a name object.  When using this function,
if the target name has a buffer associated with it, then that buffer
is used for the resulting name data; otherwise, memory is allocated for
the purpose and the name will need to be freed with `dns_name_free()`
later.

There are also multiple functions for converting name objects to text.
`dns_name_tostring()` writes the name into a buffer object, which must
have at least `DNS_NAME_MAXTEXT` bytes of available space.
`dns_name_format()` writes the name into a null-terminated
string, which must have space for at least `DNS_NAME_FORMATSIZE`
bytes.  `dns_name_tostring()` allocates memory for the text, which
must later be freed with `isc_mem_free()`.

##### Manipulating names

Several functions are provided for inspecting and modifying name objects.
These include:

* `dns_name_countlabels()` returns the number of labels in a name.
* `dns_name_getlabel()` locates a specified label in a name
  and references it in a [region object](#buffers).  In the name
  "www.example.com", label 0 is "www", label 1 is "example",
  label 2 is "com", and label 3 is the root zone.
* `dns_name_getlabelsequence` copies a specified label and a
  specified number of labels after it into a new name object.
* `dns_name_split()` separates a name into prefix and a suffix
  on a specified label boundary.  For example, "www.example.com"
  can be split into "www" and "example.com".
* `dns_name_concatenate()` concatenates a prefix and a suffix into
  a single name.

##### Comparisons

DNS name comparisons are more complex than simple string comparisons.  When
sorting names, labels at the end of the name are more significant than
labels at the beginning ("zzz.com" is less than "aaa.zzz.com").
Furthermore, it's necessary to determine relationships between names other
than simple ordering:  Whether one name is the ancestor of another, or
whether they share a common ancestor, and if so how many labels they
have in common.  The `dns_name_fullcompare()` function determines these
things.  Its return value is the relationship between two names:

        dns_namereln_t rel;
        unsigned int common;
        int order;

        /*
         * Get relationship between two names; store the sort
         * order in 'order' and the number of common labels in
         * 'common'
         */
        rel = dns_name_fullcompare(name1, name2, &order, &common);

The return value may be:

* `dns_namereln_contains`: name1 contains name2
* `dns_namereln_subdomain`: name2 contains name1
* `dns_name_commonancestor`: name1 and name2 share some labels
* `dns_name_equal`: name1 and name2 are the same

Some simpler comparison functions are provided for convenience when 
not all of this information is required:

* `dns_name_compare()`: returns the sort order of two names but
  not their relationship
* `dns_name_equal()`: returns `true` when names are equivalent
* `dns_name_caseequal()`: same as `dns_name_equal()`, but case-sensitive
* `dns_name_issubdomain()`: returns `true` if one name contains another

830
##### <a name="fixedname"></a>Fixed names
831
832
833
834
835
836
837
838

`dns_fixedname_t` is a convenience type containing a name, an offsets
table, and a dedicated buffer big enough for the longest possible DNS
name.  This allows names to be stack-allocated with minimal initialization:

        dns_fixedname_t fn;
        dns_name_t *name;

839
        name = dns_fixedname_initname(&fn);
840
841
842
843
844

`name` is now a pointer to a `dns_name` object in which a name can be
stored for the duration of this function; there is no need to initialize,
allocate, or free memory.

845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
#### <a name="rdata"></a>Rdata Classes

##### Rdataset

An rdataset (`dns_rdataset_t`) is BIND's representation of a DNS RRset,
excluding the owner name but including the type, TTL, and the contents of
each RR. The rdataset object does not hold the data itself: it is a view
that refers to data held elsewhere -- for example, in a DNS message, or in
an rbtdb (for cached or authoritative data).

It is a vaguely object-oriented polymorphic data structure, with different
implementations depending on the backing data structure that actually holds
the records. The rdataset is explicitly associated/disassociated with the
backing data structure so that it can maintain reference counts.

One important rdataset implementation is part of the red-black tree
database, implemented in `rdata.c`.

##### Rdatalist

Another backing data structure for an rdataset is the rdatalist
(`dns_rdatalist_t`) -- a linked list of rdata structures.  An rdatalist is
used to record the locations of records in a DNS message. It does not
maintain reference counts.  An rdatalist can be converted to or from an
rdataset using `dns_rdatalist_tordataset()` and
`dns_rdatalist_fromrdataset()`.

##### Rdata

See the [RRATA Types](rdata.md) document for details on type-specific
rdata conversions.

877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
#### <a name="iterators"></a>Iterators

Retrieving data from BIND databases involves the use of iterator
functions to walk from entry to entry.  Several iterator function
sets have been defined:

* `dns_dbiterator`: Walks the nodes in a database
* `dns_rdatasetiter`: Walks the RRsets in a node
* `dns_rdataset`: Walks the resource records in an RRset
* `dns_rriterator`: A combination of the previous three; walks all
   the RRs or RRsets in a database
* `dns_rbtnodechain`: Walks the nodes in a red-black tree

Each of these has a `first()`, `next()` and `current()` function; for
example, `dns_rdataset_first()`, `dns_rdataset_next()`, and
`dns_rdataset_current()`. 

The `first()` and `next()` functions move the iterator's cursor and so that
the data at a new location can be retrieved.  (Most of these can only step
by one item at a time, but `dns_rriterator` provides both `next()` and
`nextrrset()`, enabling it to step by RR or RRset.)  These functions return
`isc_result_t`, with `ISC_R_SUCCESS` indicating that there is data to
retrieve and `ISC_R_NOMORE` indicating that the iterator is finished.

The `current()` function has no return value; it simply retrieves the
data at the current cursor location.

To use an iterator, call the `first()` function, then the `current()`
function, then loop over the `next()` function until it no longer returns
success:

        for (result = dns_rdataset_first(rdataset);
             result == ISC_R_SUCCESS;
             result = dns_rdataset_next(rdataset))
        {
                dns_rdata_t rdata = DNS_RDATA_INIT;
                dns_rdataset_current(rdataset, &rdata);
                /* rdata is now populated with an RR */
        }

In some cases, calling an iterator function causes the acquisition of
918
database and/or node locks.  Rather than reacquire these locks every time
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
one of these functions is called, they are often simply held until the
iterator is destroyed.  If a caller wishes to hold an iterator open but not
use it for a while, it should call the iterator's `pause()` function (such
as `dns_dbiterator_pause()`); this will release all the locks that are
currently held by the iterator so that other threads may proceed.

#### <a name="logging"></a>Logging

The ISC logging system is designed to provide a flexible, extensible
method of writing messages, either to the system's logging facility,
directly to a file, or into the bitbucket -- usually configured per
the desires of the user of the program.

Each log message is associated with a particular category (eg, "security"
or "database") that reflects its nature, and a particular module (such as
the library's source file) that reflects its origin.  Messages are also
assigned a priority level which states how remarkable the message is;
the program's user may use this to decide how much detail is desired.

Libraries which use the ISC logging system can be linked against each
other without fear of conflict.  A program is able to select which, if
any, libraries will write log messages.

##### Fundamentals

Log messages are associated with three pieces of information that are
used to determine their disposition:  a category, a module, and a
level (aka "priority").

A category describes the conceptual nature of the message, that is,
what general aspect of the code it is concerned with.  For example,
the DNS library defines categories that include the workings of the
database as well security issues.  Macros for naming categories are
typically provided in the library's log header file, such as
`DNS_LOGCATEGORY_DATABASE` and `DNS_LOGCATEGORY_SECURITY` in `<dns/log.h>`.
The special category `ISC_LOGCATEGORY_DEFAULT` is associated with
any message that does not match a particular category (or matches a
category but not a module, as seen in the next paragraph).

A module is loosely the origin of a message.  There may not be a
one-to-one correspondence of source files with modules, but it is typical
that a module's name reflect the source file in which it is used.  So, for
example, the module identifier `DNS_LOGMODULE_RBT` would be used by
messages coming from within the `lib/dns/rbt.c` source file.

The specification of the combination of a category and a module for a
message are called the message's "category/module pair".

The level of a message is an indication of its severity.  There are
six standard logging levels, in order here from most to least severe
(least to most common):

* `ISC_LOG_CRITICAL`: An error so severe it causes the program to exit.
* `ISC_LOG_ERROR`: A very notable error, but the program can go on.
* `ISC_LOG_WARNING`: Something is probably not as it should be.
* `ISC_LOG_NOTICE`: Notable events that occur while the program runs.
* `ISC_LOG_INFO`: Statistics and routine announcements.
* `ISC_LOG_DEBUG(unsigned int level)`: Detailed debugging messages.

`ISC_LOG_DEBUG` is not quite like the others in that it takes an
argument the defines roughly how detailed the message is; a higher
level means more copious detail, so that values near 0 would be used
at places like the entry to major sections of code, while greater
numbers would be used inside loops.

The next building block of the logging system is a channel.  A channel
specifies where a message of a particular priority level should go, as
well as any special options for that destination.  There are four
basic destinations, as follows:

* `ISC_LOG_TOSYSLOG`: Send it to syslog.
* `ISC_LOG_TOFILE`: Write to a file.
* `ISC_LOG_TOFILEDESC`: Write to a (previously opened) file descriptor.
* `ISC_LOG_TONULL`: Do not write the message when selected.

A file destination names a path to a log file.  It also specifies the
maximum allowable byte size of the file before it is closed (where 0
means no limit) and the number of versions of a file to keep (where
`ISC_LOG_ROLLNEVER` means the logging system never renames the log file,
and `ISC_LOG_ROLLINFINITE` means no cap, other than integer size, on the
number of versions).  Version control is done just before a file is opened,
so a program that used it would start with a fresh log file (unless using
For faster browsing, not all history is shown. View entire blame