Commit e1c561b3 authored by Tomek Mrugalski's avatar Tomek Mrugalski 🛰

[5036] Doc updated.

parent 00d65ea6
......@@ -24,9 +24,36 @@
<para>Configuration files for DHCPv4, DHCPv6 and DDNS modules are defined
in an extended JSON format. Basic JSON is defined in <ulink
url="http://tools.ietf.org/html/rfc4627">RFC 4627</ulink>. Kea components
use a slightly modified form of JSON in that they allow shell-style
comments in the file: lines with the hash (#) character in the first column
are comment lines and are ignored.</para>
use an extended JSON with additional features allowed:
in that they allow shell-style
<itemizedlist>
<listitem>
<simpara>shell comments: any text after the hash (#)
character is ignored. Dhcp6 allows # in any column, while
Dhcp4 and Ddns require hash to be in the first
column.</simpara>
</listitem>
<listitem>
<simpara>C comments: any text after the double slashes (//)
character is ignored. We're in a process of
migrating the configuation parsers and currently only Dhcp6
supports this feature.</simpara>
</listitem>
<listitem>
<simpara>Multiline comments: any text between /* and */ is
ignored. This commenting can span multiple lines. We're in a
process of migrating the configuation parsers and currently
only Dhcp6 supports this feature.</simpara>
</listitem>
<listitem>
<simpara>File inclusion: JSON files can include other JSON
files. This can be done by using &lt;?include
"file.json"?&gt;. We're in a process of migrating the
configuation parsers and currently only Dhcp6 supports this
feature.</simpara>
</listitem>
</itemizedlist>
</para>
<para>The configuration file consists of a single object (often colloquially
called a map) started with a curly bracket. It comprises the "Dhcp4", "Dhcp6",
......
......@@ -24,6 +24,18 @@ component implementation.
@section dhcpv6ConfigParser Configuration Parsers in DHCPv6
Three minutes overview. If you are here only to learn absolute minimum about
the new parser, here's how you use it:
@code
// The following code:
json = isc::data::Element::fromJSONFile(file_name, true);
// can be replaced with this:
Parser6Context parser;
json = parser.parseFile(file_name, Parser6Context::PARSER_DHCP6);
@endcode
The common configuration parsers for the DHCP servers are located in the
src/lib/dhcpsrv/parsers/ directory. Parsers specific to the DHCPv6 component
are located in the src/bin/dhcp6/json_config_parser.cc. These parsers derive
......@@ -102,6 +114,7 @@ PHASE 1: replace isc::data::fromJSON with bison-based parser. This will allow
the JSON structures also allows us to continue using existing parsers.
Furthermore, it is possible to implement default values at this level
as simply inserting extra JSON structures in places that are necessary.
This part is covered by ticket 5036.
PHASE 2: simplify existing parsers by getting rid of the build/commit split.
Get rid of the inheritance contexts. Essentially the parser should
......@@ -114,16 +127,17 @@ PHASE 2: simplify existing parsers by getting rid of the build/commit split.
from the existing parsers and implemented in the bison parser.
It should return extra JSON elements. The details are TBD, but there is
one example for setting up an renew-timer value on the subnet level that
is ihnerited from the global ("Dhcp6") level. This phase is still a bit
loosely defined.
There is now a fully working prototype for phase 1. It introduces bison
based parser. It is essentially defined in two files: dhcp6_lexer.ll,
which defines regular expressions that are used on the input (be it
a file or a string in memory). In essence, this code is being called
repetively and each time it returns a token. This repeats until
either the parsing is complete or syntax error is encountered. For
example, for the following text:
is ihnerited from the global ("Dhcp6") level. This phase is covered by
ticket 5039.
The code change for 5036 introduces bison based parser. It is
essentially defined in two files: dhcp6_lexer.ll, which defines
regular expressions that are used on the input (be it a file or a
string in memory). In essence, this code is being called repetively
and each time it returns a token. This repeats until either the
parsing is complete or syntax error is encountered. For example, for
the following text:
@code
{
"Dhcp6":
......@@ -176,7 +190,7 @@ to the notation, it's very powerful and easy to extend. The first line defines
that dhcp6_object consists of certain tokens (DHCP6, COLON and LCURLY_BRACKET)
followed by 'global_params' expression, followed by RCURLY_BRACKET.
The global_params is defined recursively. It can either be a single 'global_param'
The 'global_params' is defined recursively. It can either be a single 'global_param'
expression, or (a shorter) global_params followed by a comma and global_param.
Bison will apply this and will be able to parse comma separated lists of
arbitrary lengths.
......@@ -223,48 +237,93 @@ is removed from the stack. At the end of parsing, there should be a single
element on the stack as the top-level parsing (syntax_map) only inserts the
MapElement object, but does not remove it.
@section dhcpv6ConfigSubParser Parsing Partial Configuation in DHCPv6
One another important capability required is the ability to parse not only the
whole configuration, but a subset of it. This is done by introducing articifical
tokens (TOPLEVEL_GENERIC_JSON and TOPLEVEL_DHCP6). The Parse6Context::parse() method
takes one parameter that specifies, whether the data to be parsed is expected
to have a generic JSON or the whole configuration (DHCP6). This is only a
proof-of-concept, but similar approach can be implemented to parse only subnets,
host reservations, options or basically any other elements. For example, to add
the ability to parse only pools, the following could be added:
tokens (e.g. TOPLEVEL_JSON and TOPLEVEL_DHCP6). For complete list of available
starting contexts, see @ref isc::dhcp::Parser6Context::ParserType. The
Parse6Context::parse() method takes one parameter that specifies, whether the
data to be parsed is expected to have a generic JSON or the whole configuration
(DHCP6). This is only a proof-of-concept, but similar approach can be implemented
to parse only subnets, host reservations, options or basically any other elements.
For example, to add the ability to parse only pools, the following could be added:
@code
start: TOPLEVEL_GENERIC_JSON map2
| TOPLEVEL_DHCP6 syntax_map
| TOPLEVEL_POOL pool_entry;
start: TOPLEVEL_GENERIC_JSON sub_json
| TOPLEVEL_DHCP6 sub_dhcp6
| TOPLEVEL_POOL6 sub_pool6;
@endcode
The code on branch trac5014 contains the code defintion and the Kea-dhcp6 updated
The code on trac5036 branch contains the code defintion and the Kea-dhcp6 updated
to use that new parser. I'm sure that parser does not cover 100% of all parameters,
but so far it is able to load all examples from doc/example/kea6. It is also
able to parser # comments (bash style, starting at the beginning or middle of
the line), // comments (C++ style, can start anywhere) or /* */ comments (C style,
the line), // comments (C++ style, can start anywhere) or / * * / comments (C style,
can span multiple lines).
This parser is currently used. See configure() method in kea_controller.cc.
There are several new unit-tests written. They're not super-extensive, but
they do cover the essentials: basic types, maps and lists encapsulating
each other in various combinations, bash, C, C++ comments. There's one
particularly useful unit-test called ParserTest.file. It loads all the
examples we have.
There are several new unit-tests written, but the code mostly reuses existing
one to verify that existing functionality was not compromised. There are
several new interesting ones, though. ParserTest.file loads all the
config file examples we have in doc/examples/kea6. This ensures that the
parser is able to load them and also checks that our examples are sane.
@section dhcp6ParserIncludes Config File Includes
The parser currently does not support file includes, but that's easy to
implement in bison-based parsers.
The new parser provides an ability to include files. The syntax was chosen
to look similar to how Apache includes PHP scripts in HTML code. This
particular syntax was chosen to emphasize that the inclusion directive
is an additional feature and not really a part of JSON syntax.
The parser's ability to parse generic JSON is somewhat fragile, because
it's more of a proof of concept rather than a solid capability. The issue
comes from the fact that if the generic json contains specific tokens that
are defined in DHCP6 nomenclature (e.g. "renew-timer"), it will interpret
it as RENEW_TIMER token rather than as STRING token. This can be solved
by having separate grammar for generic JSON if we need it. It's way
beyond the scope of proof-of-concept, though.
To include one file from another, user the following syntax:
@code
{
"Dhcp6": {
"interfaces-config": {
"interfaces": [ "*" ]},
"preferred-lifetime": 3000,
"rebind-timer": 2000,
"renew-timer": 1000,
<?include "subnets.json"?>
"valid-lifetime": 4000
}
}
@endcode
Details of the refactor of the classes derived from DhcpConfigParser is TBD.
The inclusion is implemented as a stack of files. Typically, when a single
file is parsed, the files_ (a vector of strings) and sfiles_ (a vector of FILE*)
both contain a single entry. However, when lexer detects &lt;?include "filename.json?&gt;,
it calls @ref isc::dhcp::Parser6Context::includeFile method. Up to ten
nesting levels are supported. This arbitrarily chosen limit is a protection
against recursive inclusions.
@section dhcp6ParserConflicts Avoiding syntactical conflicts in parsers
Syntactic parser has a powerful ability to not only parse the string and
check if it's a valid JSON syntax, but also check that the resulting structures
match expected syntax (if subnet6 are really an array, not a map, if
timers are expressed as integers, not as strings etc.).
However, there are times when we need to parse a string as arbitrary JSON.
For example, if we're in Dhcp6 and the config contains entries for DhcpDdns
or Dhcp4. If we were to use naive approach, the lexer would go through
that content and most likely find some tokens that are also used in Dhcp6.
for example 'renew-timer' would be detected and the parser would complain
that it was not expected. To avoid this problem, parser context was
introduced. When the syntactic parser expects certain type of content,
it calls @ref isc::dhcp::Parser6Context::enter() method to indicate what
type of content is expected. For example, when Dhcp6 parser discovers
uninteresting content like Dhcp4, it enters NO_KEYWORD mode. In this
mode, everything is parsed as generic maps, lists, integers, booleans
or strings. This results in generic JSON structures without any syntax
checking.
Details of the refactor of the classes derived from DhcpConfigParser are
documented in http://kea.isc.org/wiki/SimpleParser.
@section dhcpv6ConfigInherit DHCPv6 Configuration Inheritance
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment