Commit 36f20f1c authored by Thomas Markwalder's avatar Thomas Markwalder
Browse files

[master] Added new Stat Cmds hooks lib to open source repo

    Merge branch 'trac5589'
parents ba5fb9a8 71131063
......@@ -1400,6 +1400,8 @@ AC_CONFIG_FILES([Makefile
src/hooks/dhcp/user_chk/Makefile
src/hooks/dhcp/user_chk/tests/Makefile
src/hooks/dhcp/user_chk/tests/test_data_files_config.h
src/hooks/dhcp/stat_cmds/Makefile
src/hooks/dhcp/stat_cmds/tests/Makefile
src/lib/Makefile
src/lib/asiodns/Makefile
src/lib/asiodns/tests/Makefile
......
......@@ -780,6 +780,7 @@ INPUT = ../src/bin/agent \
../src/bin/lfc \
../src/hooks/dhcp/user_chk \
../src/hooks/dhcp/lease_cmds \
../src/hooks/dhcp/stat_cmds \
../src/lib/asiodns \
../src/lib/asiolink \
../src/lib/cc \
......
......@@ -54,6 +54,7 @@
* - @subpage hooksmgMaintenanceGuide
* - @subpage libdhcp_user_chk
* - @subpage libdhcp_lease_cmds
* - @subpage libdhcp_stat_cmds
*
* @section dhcpMaintenanceGuide DHCP Maintenance Guide
* - @subpage dhcp4
......
......@@ -8,8 +8,8 @@ dist_html_DATA = $(HTMLDOCS) kea-guide.css kea-logo-100x70.png
DOCBOOK = kea-guide.xml intro.xml quickstart.xml install.xml admin.xml config.xml
DOCBOOK += keactrl.xml dhcp4-srv.xml dhcp6-srv.xml lease-expiration.xml logging.xml
DOCBOOK += ddns.xml hooks.xml hooks-ha.xml hooks-host-cache.xml hooks-radius.xml
DOCBOOK += libdhcp.xml lfc.xml stats.xml ctrl-channel.xml faq.xml classify.xml
DOCBOOK += shell.xml agent.xml
DOCBOOK += hooks-stat-cmds.xml libdhcp.xml lfc.xml stats.xml ctrl-channel.xml
DOCBOOK += faq.xml classify.xml shell.xml agent.xml
EXTRA_DIST = $(DOCBOOK)
......
<section xml:id="hooks-stat-cmds">
<title>stat_cmds: Supplemental Statistics Commands</title>
<para>
This library provides additional statistics commands for
retrieving lease statistics from Kea DHCP servers. These commands
were added to address an issue with obtaining accurate lease
statistics in deployments running multiple Kea servers that use
shared lease back end. The in-memory statistics kept by individual
servers only track lease changes made by that server. Thus in a
deployment with multiple servers (e.g. two kea-dhcp6 servers using the
same PostgreSQL database for lease storage), these statistics are incomplete.
In Kea 1.4, the MySQL and PostgreSQL back ends were modified to track
lease allocation changes as they occur via database triggers. Additionally,
all four lease back ends were extended to support retrieving lease
statistics for all subnets, a single subnet, or a range of subnets.
Finally, this library was constructed to provide commands for retrieving
these statistics. Additional statistics commands may be added to this
library in future releases.
</para>
<para>
The commands currently provided by this library are:
<itemizedlist>
<listitem>
<para><command>stat-lease4-get</command> - fetches DHCPv4 lease statistics</para>
</listitem>
<listitem>
<para><command>stat-lease6-get</command> - fetches DHCPv6 lease statistics</para>
</listitem>
</itemizedlist>
</para>
<para>
The Stat commands library is part of the open source code and is
available to every Kea user.
</para>
<para>
All commands use JSON syntax and can be issued either directly to
the servers via the control channel (see <xref linkend="ctrl-channel"/>)
or via Control Agent (see <xref linkend="kea-ctrl-agent"/>).
</para>
<para>
This library may be loaded by both kea-dhcp4 and kea-dhcp6 servers.
It is loaded in the same way as other libraries and currently has
no parameters:
</para>
<para>
<screen>
"Dhcp6": { <userinput>
"hooks-libraries": [
{
"library": "/path/libdhcp_stat_cmds.so"
}
...
] </userinput>
}
</screen>
</para>
<para>
In a deployment with multiple Kea DHCP servers sharing a common lease
storage, it may be loaded by any or all of the servers. However, one
thing to keep in mind is that a server's response to a stat-lease{4/6}-get
command will only contain data for subnets known to that server. In
other words, if subnet does not appear in a server's configuration, it
will not retrieve statistics for it.
</para>
<section>
<title>stat-lease4-get, stat-lease6-get commands</title>
<para>
The <command>stat-lease4-get</command> and <command>stat-lease6-get</command>
commands fetch lease statistics for a range of known subnets. The
range of subnets is determined through the use of optional command
input parameters:
<itemizedlist>
<listitem>
<para><command>subnet-id</command> - ID of the subnet for which
lease statistics should be fetched. Use this to get statistics for
a single subnet. If the subnet does not exist the command result
code will be 3 (i.e. CONTROL_RESULT_EMPTY).
</para>
</listitem>
<listitem>
<para><command>subnet-range</command> - A pair of subnet IDs which
describe an inclusive range of subnets for which statistics should
be retrieved. Note that fuzzy values are supported thus allowing
for a query for statistics using approximate ID values. If the
range does not include any known subnets, the command result code
will be 3 (i.e. CONTROL_RESULT_EMPTY).
<itemizedlist>
<listitem>
<para><command>first-subnet-id</command> - ID of the first
subnet in the range.</para>
</listitem>
<listitem>
<para><command>last-subnet-id</command> - ID of the first
subnet in the range.</para>
</listitem>
</itemizedlist>
</para>
</listitem>
</itemizedlist>
The use of subnet-id and subnet-range are mutually exclusive.
If no parameters are given, the result will contain data for all
known subnets. Note that in configurations with large numbers of
subnets, this can be result in a large response.
</para>
<para>
The following command would fetch lease statistics for all known subnets from kea-dhcp4 server:
<screen>
{
"command": "stat-lease4-get"
}
</screen>
</para>
<para>
The following command would fetch lease statistcis for subnet ID 10 from kea-dhcp6 server:
<screen>
{
"command": "stat-lease6-get",
"arguments": {
"subnet-id" : 10
}
}
</screen>
</para>
<para>
The following command would fetch lease statistcis for all subnets from subnet ID 10 through 50 from kea-dhcp4 server:
<screen>
{
"command": "stat-lease4-get",
"arguments": {
"subnet-range" {
"first-subnet-id": 10,
"last-subnet-id": 50,
}
}
}
</screen>
</para>
<para>
The response to the either command will contain three elements:
<itemizedlist>
<listitem>
<para><command>result</command> - numeric value indicating the outcome of the
command where:
<itemizedlist>
<listitem><simpara><command>0</command> - command was successful</simpara></listitem>
<listitem><simpara><command>1</command> - an error occurred, an explanation will
be the "text" element</simpara></listitem>
<listitem><simpara><command>2</command> - indicates the fetch found no matching data</simpara></listitem>
</itemizedlist>
</para>
</listitem>
<listitem>
<para><command>text</command> - an explanation of the command outcome. When the command
succeeds it will contain the command name along with the number of rows returned.</para>
</listitem>
<listitem>
<para><command>arguments</command> - a map containing the data returned by the
command as the element "result-set", which patterned after SQL statement responses:
<itemizedlist>
<listitem><para><command>columns</command> - a list of text column labels.
The columns returned for DHCPv4 are:
<itemizedlist>
<listitem><simpara><command>subnet-id</command> - ID of the subnet</simpara></listitem>
<listitem><simpara><command>total-addresses</command> - total number of addresses
available for DHCPv4 management in the subnet. In other words, this is the sum of all
addresses in all the configured pools in the subnet.</simpara></listitem>
<listitem><simpara><command>assigned-addresses</command> - number of addresses
in the subnet that are currently assigned to a client.</simpara></listitem>
<listitem><simpara><command>declined-addresses</command> - number of addresses
in the subnet that are currently declined and are thus unavailable for assignment.
</simpara></listitem>
</itemizedlist></para>
</listitem>
<listitem><para>The columns returned for DHCPv6 are:
<itemizedlist>
<listitem><simpara><command>subnet-id</command> - ID of the subnet</simpara></listitem>
<listitem><simpara><command>total-nas</command> - number of NA addresses available
for DHCPv6 management in the subnet. In other words, this is the sum of all
the NA addresses in the all configured NA pools in the subnet.</simpara></listitem>
<listitem><simpara><command>assigned-nas</command> - number of NA addresses in a
the subnet that are currently assigned to a client.</simpara></listitem>
<listitem><simpara><command>declined-nas</command> - number of NA addresses that
are currently declined and are thus unavailable for assignment.</simpara></listitem>
<listitem><simpara><command>total-pds</command> - total number of prefixes available
of DHCPv6 management in the subnet. In other words, this is the sum of all prefixes
in all the configured prefix pools in the subnet.</simpara></listitem>
<listitem><simpara><command>assigned-pds</command> - number of prefixes in the subnet
that are currently assigned to a client.</simpara></listitem>
</itemizedlist></para>
</listitem>
<listitem><para><command>rows</command> - a list of rows, one per subnet ID. Each row
contains a data value for corresponding to and in the same order as each column
listed in "columns" for a given subnet.
</para></listitem>
<listitem><para><command>timestamp</command> - textual date and time the data was fetched,
expressed as GMT</para></listitem>
</itemizedlist>
</para>
</listitem>
</itemizedlist>
</para>
<para>
The response to a DHCPv4 command might look as follows:
<screen>
{
"result": 0,
"text": "stat-lease4-get: 2 rows found",
"arguments": {
"result-set": {
"columns": [ "subnet-id", "total-addresses", "assigned-addresses", "declined-addresses" ]
"rows": [
[ 10, 256, 111, 0 ],
[ 20, 4098, 2034, 4 ]
],
"timestamp": "2018-05-04 15:03:37.000000"
}
}
}
</screen>
</para>
<para>
The response to a DHCPv6 command might look as follows (subnet 10 has no
prefix pools, subnet 20 has no NA pools, and subnet 30 has both NA and PD
pools):
<screen>
{
"result": 0,
"text": "stat-lease6-get: 2 rows found",
"arguments": {
"result-set": {
"columns": [ "subnet-id", "total-nas", "assigned-nas", "declined-nas", "total-pds", "assigned-pds" ]
"rows": [
[ 10, 4096, 2400, 3, 0, 0],
[ 20, 0, 0, 0, 1048, 233 ]
[ 30, 256, 60, 0, 1048, 15 ]
],
"timestamp": "2018-05-04 15:03:37.000000"
}
}
}
</screen>
</para>
</section>
</section>
......@@ -2858,6 +2858,8 @@ both the command and the response.
<xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="hooks-host-cache.xml"/>
<xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="hooks-stat-cmds.xml"/>
</section>
<section xml:id="user-context">
......
......@@ -313,6 +313,15 @@
</simpara>
</listitem>
<listitem>
<simpara>
<command>kea-dhcp4.stat-cmds-hooks</command> - this logger is
used to log messages related to operation of the Stats Cmds hooks
library. In general these will pertain to loading and unloading
the library and the execution of commands by the library.
</simpara>
</listitem>
<listitem>
<simpara>
<command>kea-dhcp6</command> - the root logger for the DHCPv6
......@@ -437,6 +446,15 @@
</simpara>
</listitem>
<listitem>
<simpara>
<command>kea-dhcp6.stat-cmds-hooks</command> - this logger is
used to log messages related to operation of the Stats Cmds hooks
library. In general these will pertain to loading and unloading
the library and the execution of commands by the library.
</simpara>
</listitem>
<listitem>
<simpara>
<command>kea-dhcp-ddns</command> - the root logger for the
......
SUBDIRS = user_chk lease_cmds
SUBDIRS = user_chk lease_cmds stat_cmds
/stat_cmds_messages.cc
/stat_cmds_messages.h
/s-messages
/html
SUBDIRS = . tests
AM_CPPFLAGS = -I$(top_builddir)/src/lib -I$(top_srcdir)/src/lib
AM_CPPFLAGS += $(BOOST_INCLUDES)
AM_CXXFLAGS = $(KEA_CXXFLAGS)
# Define rule to build logging source files from message file
stat_cmds_messages.h stat_cmds_messages.cc: s-messages
s-messages: stat_cmds_messages.mes
$(top_builddir)/src/lib/log/compiler/kea-msg-compiler $(top_srcdir)/src/hooks/dhcp/stat_cmds/stat_cmds_messages.mes
touch $@
# Tell automake that the message files are built as part of the build process
# (so that they are built before the main library is built).
BUILT_SOURCES = stat_cmds_messages.h stat_cmds_messages.cc
# Ensure that the message file and doxygen file is included in the distribution
EXTRA_DIST = stat_cmds_messages.mes
EXTRA_DIST += stat_cmds.dox
# Get rid of generated message files on a clean
CLEANFILES = *.gcno *.gcda stat_cmds_messages.h stat_cmds_messages.cc s-messages
# convenience archive
noinst_LTLIBRARIES = libstat_cmds.la
libstat_cmds_la_SOURCES = stat_cmds.cc stat_cmds.h
libstat_cmds_la_SOURCES += stat_cmds_callouts.cc
libstat_cmds_la_SOURCES += stat_cmds_log.cc stat_cmds_log.h
libstat_cmds_la_SOURCES += version.cc
nodist_libstat_cmds_la_SOURCES = stat_cmds_messages.cc stat_cmds_messages.h
libstat_cmds_la_CXXFLAGS = $(AM_CXXFLAGS)
libstat_cmds_la_CPPFLAGS = $(AM_CPPFLAGS)
# install the shared object into $(libdir)/hooks
lib_hooksdir = $(libdir)/hooks
lib_hooks_LTLIBRARIES = libdhcp_stat_cmds.la
libdhcp_stat_cmds_la_SOURCES =
libdhcp_stat_cmds_la_LDFLAGS = $(AM_LDFLAGS)
libdhcp_stat_cmds_la_LDFLAGS += -avoid-version -export-dynamic -module
libdhcp_stat_cmds_la_LIBADD = libstat_cmds.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/dhcpsrv/libkea-dhcpsrv.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/config/libkea-cfgclient.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/cc/libkea-cc.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/hooks/libkea-hooks.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/asiolink/libkea-asiolink.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/dhcp/libkea-dhcp++.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/eval/libkea-eval.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/dhcp_ddns/libkea-dhcp_ddns.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/stats/libkea-stats.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/dns/libkea-dns++.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/cryptolink/libkea-cryptolink.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/log/libkea-log.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/util/threads/libkea-threads.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/util/libkea-util.la
libdhcp_stat_cmds_la_LIBADD += $(top_builddir)/src/lib/exceptions/libkea-exceptions.la
libdhcp_stat_cmds_la_LIBADD += $(LOG4CPLUS_LIBS)
libdhcp_stat_cmds_la_LIBADD += $(CRYPTO_LIBS)
libdhcp_stat_cmds_la_LIBADD += $(BOOST_LIBS)
This diff is collapsed.
// Copyright (C) 2018 Internet Systems Consortium, Inc. ("ISC")
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
/**
@page libdhcp_stat_cmds Kea Stat Commands Hooks Library
@section libdhcp_stat_cmdsIntro Introduction
Welcome to Kea Stat Commands Hooks Library. This documentation is addressed to
developers who are interested in the internal operation of the Stat Commands
library. This file provides information needed to understand and perhaps extend
this library.
This documentation is stand-alone: you should have read and understood <a
href="http://kea.isc.org/docs/devel/">Kea Developer's Guide</a> and in
particular its section about hooks.
@section stat_cmds Stat Commands Overview
Stat Commands (or stat_cmds) is a Hook library that can be loaded by
either kea-dhcp4 and kea-dhcp6 servers to extend them with additional
statistics mechanisms.
The initial purpose of this library is provide supplemental commands for
obtaining accurate lease statistics in deployments that share lease storage
between multiple Kea DHCP servers. It is likely that additional statistics
related commands will be added to this library in future releases as use
cases for them arise.
The library is structured around command handlers. When the library is
loaded it registers handlers for new commands. When a command
is issued (be it directly via control channel or indirectly via REST
interface from control agent), the code receives a JSON command with
parameters. The command is routed the appropriate handler, its parameters
are parsed and command executed accordingly. Lastly, a response is
constructed and shipped back to the client.
This lease statistics commands interact with both the isc::dhcp::StatsMgr
and the isc::dhcp::LeaseMgr instance. At the time of writing this text
(May, 2018), Kea supports four types of lease managers: memfile, MySQL,
PostgreSQL or Cassandra. The lease statistics commands provided by this
library provide a unified interface supported by all four of these backends.
As with other hooks, this one keeps its code in a separate namespace which
corresponds to the file name of the library: isc::stat_cmds.
For background on the design and design choices please refer to: <a
href="http://kea.isc.org/wiki/SharedLeaseStorageStats">Shared Lease Stats Design</a>
@section stat_cmdsCode Stat Commands Code Overview
Library operation starts with Kea calling the load() function (file
stat_cmds_callouts.cc). This function registers the command callout
functions for each of the libraries commands. For a list,
see @ref isc::stat_cmds::StatCmds class documentation. This class uses
the Pimpl design pattern, and thus the actual implementation is hidden
in @ref isc::stat_cmds::LeaseStatCmdsImpl.
Unlike similar libraries, the Pimpl class is differentiated from the
the StatCmds class by the prefix "Lease" and it is instantiated in the outer
handler (e.g. @ref isc::stat_cmds::StatCmds::statLease4GetHandler) rather than
the StatCmds constructor. This was done in the event that commands, unrelated
to lease statistics, are added to this library and that would be better served
by Pimpl derviations specific to them.
@subsection stat_cmdsLeaseStatCode Lease Stat Commands Code Overview
There are two shared lease statistic commands, "stat-lease4-get" and "stat-lease6-get".
Both of these support the same input parameters and have their own command handlers:
- @ref isc::stat_cmds::LeaseStatCmdsImpl::statLease4GetHandler (stat-lease4-get)
- @ref isc::stat_cmds::LeaseStatCmdsImpl::statLease6GetHandler (stat-lease6-get)
Briefly, the commands are intended to fetch the lease statistics per subnet
for the range of subnets described by the input parameters. JSON text structure
of the commands is as follows:
DHCPv4 command:
@code
{
"command": "stat-lease4-get",
"arguments": {
"subnet-id": x
"subnet-range": {
"first-subnet-id": x,
"last-subnet-id": y
}
}
}
where subnet-id and subnet-range are optional and mutually exclusive
@endcode
DHCPv6 command:
@code
{
"command": "stat-lease6-get",
"arguments": {
"subnet-id": x
"subnet-range": {
"first-subnet-id": x,
"last-subnet-id": y
}
}
}
where subnet-id and subnet-range are optional and mutually exclusive
@endcode
Command handling
for both commands is symetrical consists of the following steps:
-# The input parameters (if any) are parsed. Invalid values or
combinations if detected result in generating a CONTROL_RESULT_ERROR
response to the client. (See @ref isc::stat_cmds::LeaseStatCmdsImpl::getParameters
and @ref isc::stat_cmds::LeaseStatCmdsImpl::Parameters)
-# The parameters are used to identify the range of configured subnets
to include in the response. This is done by querying subnet configuration
housed by @ref isc::dhcp::CfgMgr. If the range contains no known subnets then a
CONTROL_RESULT_EMPTY response is sent back to the client.
-# The initial result-set response is constructed. Essentially this is
an @ref isc::data::Element tree, which converted to JSON would appear as follows:
@code
"result-set": {
"timestamp": "2018-03-22 09:43:30.815371",
"columns": ["<label-1>, <label-2>, ... ],
"rows": []
}
@endcode
-# The acutal lease statistics are then retrieved from @ref isc::dhcp::LeaseMgr instance
by invoking of the three functions based on the input parameters:
DHCPv4 functions:
- @ref isc::dhcp::LeaseMgr::startLeaseStatsQuery4
- @ref isc::dhcp::LeaseMgr::startSubnetLeaseStatsQuery4
- @ref isc::dhcp::LeaseMgr::startSubnetRangeLeaseStatsQuery4
DHCPv6 functions:
- @ref isc::dhcp::LeaseMgr::startLeaseStatsQuery6
- @ref isc::dhcp::LeaseMgr::startSubnetLeaseStatsQuery6
- @ref isc::dhcp::LeaseMgr::startSubnetRangeLeaseStatsQuery6
-# Iterate over the range of qualifying subnets adding a row of statistics for
each subnet to the result-set. Each row combines the subnet total(s) from isc::dhcp::StatsMgr
with the type and state counts from the lease query results. For subnets with no query
data (i.e. no leases), their rows have non-zero values for totals only.
-# Finally, a CONTROL_RESULT_SUCCESS repsonse is generated containing the populated result-set.
A DHCPv4 response might look like this:
@code
{
"result-set\": {
"columns\": [
"subnet-id", "total-addreses",
"assigned-addreses", "declined-addreses"
],
"rows\": [
[ 30, 256, 100, 2 ],
[ 40, 16, 0, 0 ],
[ 50, 256, 35, 0 ],
],
timestamp\": \"2018-05-04 15:03:37.000000\" }
}
@endcode
and DHCPv6 response might look like this:
@code
{
"result-set": {
"columns": [
"subnet-id", "total-nas", "assigned-nas",
"declined-nas", "total-pds", "assigned-pds"
],
"rows": [
[ 30, 16, 6, 0, 65536, 400 ],
[ 40, 16777216, 989837, 0, 0, 0 ]
],
"timestamp": "2018-05-04 15:03:37.000000" }
}
@endcode
*/
// Copyright (C) 2018 Internet Systems Consortium, Inc. ("ISC")
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
#ifndef STAT_CMDS_H
#define STAT_CMDS_H
#include <hooks/hooks.h>
#include <boost/shared_ptr.hpp>
namespace isc {
namespace stat_cmds {
/// @brief Forward declaration of implementation class.
class StatCmdsImpl;
/// @brief Implements the logic for processing commands pertaining to
/// stat manipulation.
///