ctrl-channel.xml 25.3 KB
Newer Older
1 2 3 4 5 6 7 8
<!--
 - Copyright (C) 2015-2018 Internet Systems Consortium, Inc. ("ISC")
 -
 - This Source Code Form is subject to the terms of the Mozilla Public
 - License, v. 2.0. If a copy of the MPL was not distributed with this
 - file, You can obtain one at http://mozilla.org/MPL/2.0/.
-->

9 10
<!-- Converted by db4-upgrade version 1.1 -->
<chapter xmlns="http://docbook.org/ns/docbook" version="5.0" xml:id="ctrl-channel">
11
    <title>Management API</title>
12

13
    <para>A classic approach to daemon configuration assumes that
14 15
    the server's configuration is stored in configuration files
    and, when the configuration is changed, the daemon is restarted.
16 17
    This approach has the significant disadvantage of introducing periods
    of downtime, when client traffic is not handled. Another risk
18 19
    is that if the new configuration is invalid for whatever reason,
    the server may refuse to start, which will further extend the
20
    downtime period until the issue is resolved.</para>
21

22
    <para>To avoid such problems, the DHCPv4, DHCPv6 and D2 servers
23 24
    include support for a mechanism that allows
    on-line reconfiguration without requiring server shutdown.
25 26
    Both servers can be instructed to open control sockets, which
    is a communication channel. The server is able to receive
27
    commands on that channel, act on them and report back status.</para>
28

29
    <para>The DHCPv4, DHCPv6 and D2 servers receive commands over the
30
    unix domain sockets. The details how to configure these sockets,
31
    see <xref linkend="dhcp4-ctrl-channel"/> and <xref linkend="dhcp6-ctrl-channel"/>. While it is possible control
32
    the servers directly using unix domain sockets it requires that
33
    the controlling client be running on the same machine as
34 35 36 37 38 39 40
    the server. In order to connect remotely SSH is usually used to
    connect to the controlled machine.</para>

    <para>The network administrators usually prefer using some form
    of a RESTful API to control the servers, rather than using unix
    domain sockets directly. Therefore, as of Kea 1.2.0 release,
    Kea includes a new component called Control Agent (or CA) which
41
    exposes a RESTful API to the controlling clients and can forward
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58
    commands to the respective Kea services over the unix domain
    sockets. The CA configuration has been described in
    <xref linkend="agent-configuration"/>.</para>

    <para>The HTTP requests received by the CA contain the control
    commands encapsulated within HTTP requests. Simply speaking,
    the CA is responsible for stripping the HTTP layer from the
    received commands and forwarding the commands in a JSON format
    over the unix domain sockets to respective services. Because the
    CA receives commands for all services it requires additional
    "forwarding" information to be included in the client's messages.
    This "forwarding" information is carried within the
    <command>service</command> parameter of the received command.
    If the <command>service</command> parameter is not included or if
    the parameter is a blank list the CA will assume that the control
    command is targetted at the CA itself and will try to handle
    it on its own.
59 60
    </para>

61 62 63 64 65 66 67 68 69
    <para>Control connections over both HTTP and unix domain sockets are
    guarded with timeouts. The default timeout value is set to 10s
    and is not configurable. The timeout configuration will be
    implemented in the future.
    </para>

    <note>
      <simpara>Kea 1.2.0 release and earlier had a limitation of 64kB
      on the maximum size of a command and a response sent over the unix
70
      domain socket. This limitation has been removed in Kea 1.3.0
71 72 73 74
      release.
      </simpara>
    </note>

75
    <section xml:id="ctrl-channel-syntax">
76 77 78 79
    <title>Data Syntax</title>
    <para>Communication over the control channel is conducted using JSON
    structures. If configured, Kea will open a socket and listen
    for incoming connections. A process connecting to this socket
80 81 82 83 84
    is expected to send JSON commands structured as follows:

<screen>
{
    "command": "foo",
85
    "service": [ "dhcp4" ]
86
    "arguments": {
87 88 89
        "param1": "value1",
        "param2": "value2",
        ...
90 91 92 93
    }
}
</screen>

94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111
    The same command sent over the RESTful interface to the CA will have
    the following structure.

<screen>
    POST / HTTP/1.1\r\n
    Content-Type: application/json\r\n
    Content-Length: 147\r\n\r\n
    {
        "command": "foo",
        "service": [ "dhcp4" ]
        "arguments": {
            "param1": "value1",
            "param2": "value2",
            ...
        }
    }
</screen>

112 113 114
    <command>command</command> is the name of command to execute and
    is mandatory. <command>arguments</command> is a map of parameters
    required to carry out the given command.  The exact content and
115
    format of the map is command specific.</para>
116

117 118 119 120 121 122 123
    <para>
      <command>service</command> is a list of the servers at which the control
      command is targetted. In the example above, the control command is
      targetted at the DHCPv4 server. In most cases, the CA will simply forward this
      command to the DHCPv4 server for processing via unix domain socket.
      Sometimes, the command including a service value may also be processed by the
      CA, if the CA is running a hooks library which handles such command for the
124
      given server. As an example, the hooks library loaded by the CA
125 126 127 128 129
      may perform some operations on the database (like adding host reservations,
      modifying leases etc.). An advantage of performing DHCPv4 specific
      administrative operations in the CA rather than forwarding it to
      the DHCPv4 server is the ability to perform these operations without
      disrupting the DHCPv4 service (DHCPv4 server doesn't have to stop
Josh Soref's avatar
Josh Soref committed
130
      processing DHCP messages to apply changes to the database). Nevertheless,
131 132
      these situations are rather rare and, in most cases, when the
      <command>service</command> parameter contains a name of the service
133 134 135 136
      the commands are simply forwarded by the CA. The forwarded command
      includes the <command>service</command> parameter but this parameter
      is ignored by the receiving server. This parameter is only meaningful
      to the CA.
137 138 139 140 141 142 143 144 145 146 147 148
    </para>

    <para>
      If the command received by the CA does not include a <command>service</command>
      parameter or this list is empty, the CA will simply process this message
      on its own. For example, the <command>config-get</command> command which
      doesn't include service parameter will return Control Agent's own
      configuration. The <command>config-get</command> including a service
      value "dhcp4" will be forwarded to the DHCPv4 server and will return
      DHCPv4 server's configuration and so on.
    </para>

149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171
    <para>
      The following list contains a mapping of the values carried within the
      <command>service</command> parameter to the servers to which the commands
      are forwarded:

      <itemizedlist>
        <listitem>
          <simpara><command>dhcp4</command> - the command is forwarded to the
          <command>kea-dhcp4</command> server,</simpara>
        </listitem>

        <listitem>
          <simpara><command>dhcp6</command> - the command is forwarded to the
          <command>kea-dhcp6</command> server,</simpara>
        </listitem>

        <listitem>
          <simpara><command>d2</command> - the command is forwarded to the
          <command>kea-d2</command> server.</simpara>
        </listitem>
      </itemizedlist>
    </para>

172 173
    <para>The server processing the incoming command will send a response of
    the form:
174 175
<screen>
{
176
    "result": 0|1|2|3,
177 178
    "text": "textual description",
    "arguments": {
179 180 181
        "argument1": "value1",
        "argument2": "value2",
        ...
182 183 184 185
    }
}
</screen>
    <command>result</command> indicates the outcome of the command. A value of 0
186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205
    means success while any non-zero value designates an error or at least a
    failure to complete the requested action. Currently 1 is used as a generic
    error, 2 means that a command is not supported and 3 means that the
    requested operation was completed, but the requested object was not
    found. Additional error codes may be added in the future. For example a well
    formed command that requests a subnet that exists in server's configuration
    would return result 0. If the server encounters an error condition, it would
    return 1. If the command was asking for IPv6 subnet, but was sent to DHCPv4
    server, it would return 2. If the query was asking for a subnet-id and there
    is no subnet with such id, the result would be set to 3.</para>

    <para>
    The <command>text</command> field typically appears when result is non-zero
    and contains a description of the error encountered, but it often also appears
    for successful outcomes. The exact text is command specific, but in general
    uses plain English to describe the outcome of the command.
    <command>arguments</command> is a map of additional data values
    returned by the server which is specific to the command issued. The map is
    may be present, but that depends on specific command.
    </para>
206 207 208 209 210

    <note>
      <simpara>
        When sending commands via Control Agent, it is possible to specify
        multiple services at which the command is targetted. CA will forward this
211
        command to each service individually. Thus, the CA response to the
212 213 214 215
        controlling client will contain an array of individual responses.
      </simpara>
    </note>

216 217
    </section>

218
    <section xml:id="ctrl-channel-client">
219
    <title>Using the Control Channel</title>
220

221 222
    <para>Kea development team is actively working on providing client applications
    which can be used to control the servers. These applications are, however, in the
Josh Soref's avatar
Josh Soref committed
223
    early stages of development and as of Kea 1.2.0 release have certain limitations.
224
    The easiest way to start interacting with the control API is to use common Unix/Linux tools
225 226 227
    such as <command>socat</command> and <command>curl</command>.</para>

    <para>In order to control the given Kea service via unix domain socket, use
228
    <command>socat</command> in interactive mode as follows:
229 230
<screen>
$ socat UNIX:/path/to/the/kea/socket -
231 232 233 234 235
</screen>
    or in batch mode, include the "ignoreeof" option as shown below to ensure
    socat waits long enough for the server to respond:
<screen>
$ echo "{ some command...}" | socat UNIX:/path/to/the/kea/socket -,ignoreeof
236
</screen>
237 238
where <command>/path/to/the/kea/socket</command> is the path specified in the
<command>Dhcp4/control-socket/socket-name</command> parameter in the Kea
239
configuration file. Text passed to <command>socat</command>
240 241 242
will be sent to Kea and the responses received from Kea printed to standard
    output. This approach communicates with the specific server directly and
    bypasses Control Agent.</para>
243

Josh Soref's avatar
Josh Soref committed
244
    <para>It is also easy to open UNIX socket programmatically. An example of
245
    such a simplistic client written in C is available in the Kea Developer's
246 247
    Guide, chapter Control Channel Overview, section Using Control Channel.</para>

248 249
    <para>In order to use Kea's RESTful API with <command>curl</command> you may
    use the following:
250 251 252 253 254 255 256 257
<screen>
$ curl -X POST -H "Content-Type: application/json" -d '{ "command": "config-get", "service": [ "dhcp4" ] }' http://ca.example.org:8000/
</screen>

    This assumes that the Control Agent is running on host
    <filename>ca.example.org</filename> and runs RESTful service on port 8000.
    </para>

258 259
    </section>

260
    <section xml:id="commands-common">
261
      <title>Commands Supported by Both the DHCPv4 and DHCPv6 Servers</title>
262

263
      <section xml:id="command-build-report">
Francis Dupont's avatar
Francis Dupont committed
264 265 266 267 268 269 270 271 272 273 274 275 276 277 278
        <title>build-report</title>
        <para>
          The <emphasis>build-report</emphasis> command returns
          on the control channel what the command line
          <command>-W</command> argument displays, i.e. the embedded
          content of the <filename>config.report</filename> file.
          This command does not take any parameters.
        </para>
<screen>
{
    "command": "build-report"
}
</screen>
      </section> <!-- end of command-build-report -->

279
      <section xml:id="command-config-get">
280 281 282 283 284 285
        <title>config-get</title>

        <para>The <emphasis>config-get</emphasis> command retrieves the current
        configuration used by the server. This command does not take any
        parameters. The configuration returned is roughly equal to the
        configuration that was loaded using -c command line option during server
286
        start-up or later set using config-set command. However, there may be
287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303
        certain differences. Comments are not retained. If the original
        configuration used file inclusion, the returned configuration will
        include all parameters from all the included files.</para>

        <para> Note that returned configuration is not redacted, i.e. it will
        contain database passwords in plain text if those were specified in the
        original configuration. Care should be taken to not expose the command
        channel to unprivileged users.</para>

        <para>
          An example command invocation looks like this:
<screen>
{
    "command": "config-get"
}
</screen>
        </para>
Francis Dupont's avatar
Francis Dupont committed
304 305
      </section> <!-- end of command-config-get -->

306
      <section xml:id="command-config-reload">
307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334
        <title>config-reload</title>

        <para>The <emphasis>config-reload</emphasis> command instructs
        Kea to load again the configuration file that was used
        previously. This operation is useful if the configuration file
        has been changed by some external sources. For example, a
        sysadmin can tweak the configuration file and use this command
        to force Kea pick up the changes.</para>

        <para>Caution should be taken when mixing this with config-set
        commands. Kea remembers the location of the configuration file
        it was started with. This configuration can be significantly
        changed using config-set command. When config-reload is issued
        after config-set, Kea will attempt to reload its original
        configuration from the file, possibly losing all changes
        introduced using config-set or other commands.</para>

        <para><emphasis>config-reload</emphasis> does not take any parameters.
        An example command invocation looks like this:
<screen>
{
    "command": "config-reload"
}
</screen>
        </para>
      </section> <!-- end of command-config-reload -->


335
      <section xml:id="command-config-test">
Francis Dupont's avatar
Francis Dupont committed
336 337 338 339 340 341 342 343 344
        <title>config-test</title>

      <para>
     The <emphasis>config-test</emphasis> command instructs the server to check
     whether the new configuration supplied in the command's arguments can
     be loaded. The supplied configuration is expected to be the full
     configuration for the target server along with an optional Logger
     configuration. As for the <command>-t</command> command some sanity checks
     are not performed so it is possible a configuration which successfully
345
     passes this command will still fail in <command>config-set</command>
Francis Dupont's avatar
Francis Dupont committed
346 347 348 349 350 351 352
     command or at launch time.
     The structure of the command is as follows:
      </para>
<screen>
{
    "command": "config-test",
    "arguments":  {
353
        "&lt;server&gt;": {
Francis Dupont's avatar
Francis Dupont committed
354 355 356 357 358 359 360
        },
        "Logging": {
        }
     }
}
</screen>
      <para>
361
    where &lt;server&gt; is the configuration element name for a given server
Francis Dupont's avatar
Francis Dupont committed
362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384
    such as "Dhcp4" or "Dhcp6".  For example:
      </para>
<screen>
{
    "command": "config-test",
    "arguments":  {
        "Dhcp6": {
            :
        },
        "Logging": {
            :
        }
     }
}
</screen>
      <para>
        The server's response will contain a numeric code, "result" (0 for success,
    non-zero on failure), and  a string, "text", describing the outcome:
<screen>
    {"result": 0, "text": "Configuration seems sane..." }

    or

385
    {"result": 1, "text": "unsupported parameter: BOGUS (&lt;string&gt;:16:26)" }
Francis Dupont's avatar
Francis Dupont committed
386 387 388
</screen>
      </para>
    </section> <!-- end of command-config-test -->
389

390
      <section xml:id="command-config-write">
391 392 393 394 395
        <title>config-write</title>

        <para>The <emphasis>config-write</emphasis> command instructs Kea server
        to write its current configuration to a file on disk. It takes one
        optional argument called <emphasis>filename</emphasis> that specifies
Francis Dupont's avatar
Francis Dupont committed
396
        the name of the file to write configuration to. If not specified, the
397 398 399
        name used when starting Kea (passed as -c argument) will be
        used. If relative path is specified, Kea will write its files
        only in the directory it is running.</para>
400 401 402 403 404
        <para>
          An example command invocation looks like this:
<screen>
{
    "command": "config-write",
Tomek Mrugalski's avatar
Tomek Mrugalski committed
405
    "arguments": {
406 407 408 409 410
        "filename": "config-modified-2017-03-15.json"
    }
}
</screen>
        </para>
Francis Dupont's avatar
Francis Dupont committed
411
      </section> <!-- end of command-config-write -->
412

413
      <section xml:id="command-leases-reclaim">
414
        <title>leases-reclaim</title>
415
        <para>
416
          The <emphasis>leases-reclaim</emphasis> command instructs the server to
417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433
          reclaim all expired leases immediately. The command has the following
          JSON syntax:
<screen>
{
    "command": "leases-reclaim",
    "arguments": {
        "remove": true
    }
}
</screen>
        </para>

        <para>The <emphasis>remove</emphasis> boolean parameter is mandatory
        and it indicates whether the reclaimed leases should be removed from
        the lease database (if true), or they should be left in the
        <emphasis>expired-reclaimed</emphasis> state (if false). The latter
        facilitates lease affinity, i.e. ability to re-assign expired lease to
434 435
        the same client which used this lease before. See
        <xref linkend="lease-affinity"/> for the details. Also, see
436 437 438 439
        <xref linkend="lease-reclamation"/> for the general information
        about the processing of expired leases (leases reclamation).</para>
      </section>

440
      <section xml:id="command-libreload">
441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457
        <title>libreload</title>

        <para>
          The <emphasis>libreload</emphasis> command will first unload and then
          load all currently loaded hook libraries.  This is primarily intended
          to allow one or more hook libraries to be replaced with newer versions
          without requiring Kea servers to be reconfigured or restarted.  Note
          the hook libraries will be passed the same parameter values (if any)
          they were passed when originally loaded.
<screen>
{
    "command": "libreload",
    "arguments": { }
}
</screen>
       </para>
       <para>
Francis Dupont's avatar
Francis Dupont committed
458
         The server will respond with a result of 0 indicating success, or 1
459 460 461 462
         indicating a failure.
       </para>
     </section> <!-- end of command-libreload -->

463
      <section xml:id="command-list-commands">
464
      <title>list-commands</title>
465 466

      <para>
467 468 469
        The <emphasis>list-commands</emphasis> command retrieves a list of all
        commands supported by the server. It does not take any arguments.
        An example command may look like this:
470 471 472 473 474 475 476 477
<screen>
{
    "command": "list-commands",
    "arguments": { }
}
</screen>
      </para>
      <para>
478 479 480
        The server will respond with a list of all supported commands. The
        arguments element will be a list of strings. Each string will convey
        one supported command.
481 482 483
      </para>
    </section> <!-- end of command-list-commands -->

484
    <section xml:id="command-config-set">
485
      <title>config-set</title>
486 487

      <para>
488
    The <emphasis>config-set</emphasis> command instructs the server to replace
489 490 491 492 493 494 495 496 497
    its current configuration with the new configuration supplied in the
    command's arguments. The supplied configuration is expected to be the full
    configuration for the target server along with an optional Logger
    configuration.  While optional, the Logger configuration is highly
    recommended as without it the server will revert to its default logging
    configuration. The structure of the command is as follows:
      </para>
<screen>
{
498
    "command": "config-set",
499
    "arguments":  {
500
        "&lt;server&gt;": {
501 502 503 504 505 506 507
        },
        "Logging": {
        }
     }
}
</screen>
      <para>
508
    where &lt;server&gt; is the configuration element name for a given server
509 510 511 512
    such as "Dhcp4" or "Dhcp6".  For example:
      </para>
<screen>
{
513
    "command": "config-set",
514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530
    "arguments":  {
        "Dhcp6": {
            :
        },
        "Logging": {
            :
        }
     }
}
</screen>
      <para>
    If the new configuration proves to be invalid the server will retain
    its current configuration.  Please note that the new configuration is
    retained in memory only.  If the server is restarted or a configuration
    reload is triggered via a signal, the server will use the configuration
    stored in its configuration file.

531
        The server's response will contain a numeric code, "result" (0 for success,
532 533 534 535 536 537
    non-zero on failure), and  a string, "text", describing the outcome:
<screen>
    {"result": 0, "text": "Configuration successful." }

    or

538
    {"result": 1, "text": "unsupported parameter: BOGUS (&lt;string&gt;:16:26)" }
539 540
</screen>
      </para>
541
    </section> <!-- end of command-config-set -->
542

543
    <section xml:id="command-shutdown">
544
      <title>shutdown</title>
545 546

      <para>
547 548 549 550
        The <emphasis>shutdown</emphasis> command instructs the server to initiate
        its shutdown procedure. It is the equivalent of sending a SIGTERM signal
        to the process. This command does not take any arguments.  An example
        command may look like this:
551 552
<screen>
{
553
    "command": "shutdown"
554 555 556 557
}
</screen>
      </para>
      <para>
558 559
        The server will respond with a confirmation that the shutdown procedure
        has been initiated.
560 561 562
      </para>
    </section> <!-- end of command-shutdown -->

563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598
    <section id="command-dhcp-disable">
      <title>dhcp-disable</title>
      <para>
        The <emphasis>dhcp-disable</emphasis> command globally disables the DHCP
        service. The server continues to operate, but it drops all received DHCP
        messages. This command is useful when the server's maintenance requires that
        the server temporarily stops allocating new leases and renew existing leases.
        It is also useful in failover like configurations during a synchronization of
        the lease databases at startup or recovery after a failure. The optional parameter
        <emphasis>max-period</emphasis> specifies the time in seconds after which the
        DHCP service should be automatically re-enabled if the
        <emphasis>dhcp-enable</emphasis> command is not sent before this time elapses.
      </para>
<screen>
{
    "command": "dhcp-disable",
    "arguments": {
        "max-period": 20
    }
}
</screen>
    </section> <!-- end of command-dhcp-disable -->

    <section id="command-dhcp-enable">
      <title>dhcp-enable</title>
      <para>
        The <emphasis>dhcp-enable</emphasis> command globally enables the DHCP
        service.
      </para>
<screen>
{
    "command": "dhcp-enable"
}
</screen>
    </section> <!-- end of command-dhcp-disable -->

599
      <section xml:id="command-version-get">
Francis Dupont's avatar
Francis Dupont committed
600 601
        <title>version-get</title>
        <para>
602 603 604 605
          The <emphasis>version-get</emphasis> command returns an
          extended information about Kea version. It is the same
          information as Kea caled with <command>-V</command> command
          line argument. This command does not take any parameters.
Francis Dupont's avatar
Francis Dupont committed
606 607 608 609 610 611 612
        </para>
<screen>
{
    "command": "version-get"
}
</screen>
      </section> <!-- end of command-version-get -->
613

614 615
    </section> <!-- end of commands supported by both servers -->

616 617 618 619 620 621 622 623 624 625 626
    <section>
      <title>Commands Supported by D2 Server</title>
      <para>The D2 server supports only a subset of DHCPv4 / DHCPv6 server
      commands:
      <itemizedlist>
        <listitem>
          <simpara>build-report</simpara>
        </listitem>
        <listitem>
          <simpara>config-get</simpara>
        </listitem>
627 628 629
        <listitem>
          <simpara>config-set</simpara>
        </listitem>
630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648
        <listitem>
          <simpara>config-test</simpara>
        </listitem>
        <listitem>
          <simpara>config-write</simpara>
        </listitem>
        <listitem>
          <simpara>list-commands</simpara>
        </listitem>
        <listitem>
          <simpara>shutdown</simpara>
        </listitem>
        <listitem>
          <simpara>version-get</simpara>
        </listitem>
      </itemizedlist>
      </para>
    </section>

649
    <section xml:id="agent-commands">
650 651 652 653 654 655 656 657 658 659 660 661
      <title>Commands Supported by Control Agent</title>
      <para>The following commands listed in <xref linkend="commands-common"/>
      are also supported by the Control Agent, i.e. when the
      <command>service</command> parameter is blank the commands are handled
      by the CA and they relate to the CA process itself:
      <itemizedlist>
        <listitem>
          <simpara>build-report</simpara>
        </listitem>
        <listitem>
          <simpara>config-get</simpara>
        </listitem>
662 663 664
        <listitem>
          <simpara>config-set</simpara>
        </listitem>
665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683
        <listitem>
          <simpara>config-test</simpara>
        </listitem>
        <listitem>
          <simpara>config-write</simpara>
        </listitem>
        <listitem>
          <simpara>list-commands</simpara>
        </listitem>
        <listitem>
          <simpara>shutdown</simpara>
        </listitem>
        <listitem>
          <simpara>version-get</simpara>
        </listitem>
      </itemizedlist>
      </para>
    </section>

684
  </chapter>