[3720] Add sections about LFC to the developes and administrators guides

Merge branch 'trac3720'

Conflicts:
	ChangeLog

ChangeLog

+912.	[doc]		sar
+	Added sections on LFC to the administrators and developers
+	guides.
+	(Trac #3720, git )
+
 911.	[func]*		marcin
 	libdhcp++: the C++ objects representing network interfaces
 	(Iface objects) are now non-copyable. As a result, the API

doc/devel/mainpage.dox

-// Copyright (C) 2012-2014  Internet Systems Consortium, Inc. ("ISC")
+// Copyright (C) 2012-2015  Internet Systems Consortium, Inc. ("ISC")
 //
 // Permission to use, copy, modify, and/or distribute this software for any
 // purpose with or without fee is hereby granted, provided that the above
@@ -78,6 +78,9 @@
  *   - @subpage d2TransDetail
  *   - @subpage d2StateModel
  *   - @subpage d2TransExecExample
+ * - @subpage lfc
+ *   - @subpage lfcProcessing
+ *   - @subpage lfcFiles
  * - @subpage libdhcp
  *   - @subpage libdhcpIntro
  *   - @subpage libdhcpRelay

doc/guide/Makefile.am

@@ -7,7 +7,7 @@ dist_html_DATA = $(HTMLDOCS) kea-guide.css
 
 DOCBOOK = kea-guide.xml intro.xml quickstart.xml install.xml admin.xml config.xml
 DOCBOOK += keactrl.xml dhcp4-srv.xml dhcp6-srv.xml logging.xml ddns.xml hooks.xml
-DOCBOOK += libdhcp.xml
+DOCBOOK += libdhcp.xml lfc.xml
 
 EXTRA_DIST = $(DOCBOOK)
 DISTCLEANFILES = $(HTMLDOCS) $(DOCS) kea-messages.xml

doc/guide/intro.xml

@@ -146,6 +146,17 @@
             </simpara>
           </listitem>
 
+          <listitem>
+            <simpara>
+              <command>kea-lfc</command> &mdash;
+              This process removes redundant information from the files used
+              to provide persistent storage for the memfile data base backend.
+              The service is written to run as a stand alone process.  While it
+              can be started externally it should be started by the Kea DHCP
+              servers.
+            </simpara>
+          </listitem>
+
           <listitem>
             <simpara>
               <command>perfdhcp</command> &mdash;

doc/guide/kea-guide.xml

@@ -69,6 +69,8 @@
 
   <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="ddns.xml" />
 
+  <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="lfc.xml" />
+
   <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="hooks.xml" />
 
   <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="libdhcp.xml" />

doc/guide/lfc.xml

+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [
+<!ENTITY mdash  "&#x2014;" >
+]>
+
+  <chapter id="kea-lfc">
+    <title>The LFC process</title>
+
+    <section id="kea-lfc-overview">
+      <title>Overview</title>
+      <para><command>kea-lfc</command> is a service process that removes
+      redundant infomation from the files used to provide persistent storage
+      for the memfile data base backend. This service is written to run as a
+      stand alone process.  
+      </para>
+      <para>While <command>kea-lfc</command> can be started externally, there is
+      usually no need to do this.  <command>kea-lfc</command> is run on a periodic
+      basis by the Kea DHCP servers.
+      </para>
+      <para>The process operates on a set of files, using them for input and output
+      of the lease entries and to indicate where it is in the process in case of an
+      interruption.  Currently the caller must supply names for all of the files, in
+      the future this requirement may be relaxed with the process getting the names
+      from either the config file or from defaults.
+      </para>
+    </section>
+
+    <section id="kea-lfc-usage">
+      <title>Command Line Options</title>
+      <para><command>kea-lfc</command> is run as follows:
+<screen>
+kea-lfc [-4 | -6] -c config-file -p pid-file -x previous-file -i copy-file -o output-file -f finish-file
+</screen>
+      </para>
+
+      <para>The argument <command>-4</command> or <command>-6</command> selects the protocol
+      version of the lease files.
+      </para>
+
+      <para>The <command>-c</command> argument specifies the configuration file.  This is
+      required, but not currently used by the process.
+      </para>
+
+      <para>The <command>-p</command> argument specifies the PID file. When the
+      <command>kea-lfc</command> process starts it attempts to determine if another
+      instance of the process is already running by examining the pid file.  If one
+      is already running the new process is terminated.  If one isn't running it writes
+      its pid into the pid file.
+      </para>
+
+      <para>The other filenames specify where the <command>kea-lfc</command> process
+      should look for input, write its output and use for bookkeeping.
+
+        <itemizedlist>
+          <listitem><simpara>
+            <command>previous</command> &mdash;
+            When <command>kea-lfc</command> starts this
+            is the result of any previous run of <command>kea-lfc</command>.
+            When <command>kea-lfc</command> finishes it is the result of this run.
+            If <command>kea-lfc</command> is interrupted before compelting
+            this file may not exist.
+          </simpara></listitem>
+
+          <listitem><simpara>
+            <command>input</command> &mdash;
+            Before the DHCP server invokes <command>kea-lfc</command> it will
+            move the current lease file here and then call <command>kea-lfc</command>
+            with this file.
+          </simpara></listitem>
+
+          <listitem><simpara>
+            <command>output</command> &mdash;
+            The temporary file <command>kea-lfc</command> should use to write the leases.
+            Upon completion of writing this this file it will be moved to the finish file
+           (see below).
+          </simpara></listitem>
+
+          <listitem><simpara>
+            <command>finish</command> &mdash;
+            Another temporary file <command>kea-lfc</command> uses for bookkeeping.  When
+            <command>kea-lfc</command> completes writing the outputfile it moves it to this
+            file name.  After <command>kea-lfc</command> finishes deleting the other files
+            (previous and input) it moves this file to previous lease file.  By moving the
+            files in this fashion the <command>kea-lfc</command> and the DHCP server
+            processes can determine the correct file to use even if one of the
+            processes was interrupted before completing its task.
+          </simpara></listitem>
+
+        </itemizedlist>
+      </para>
+
+      <para>There are several additional arguments mostly for debugging purposes.
+      <command>-d</command> Sets the logging level to debug. <command>-v</command> and
+      <command>-V</command> print out version stamps with <command>-V</command> providing
+      a longer form.  <command>-h</command> prints out the usage string.
+      </para>
+
+    </section>
+  </chapter>

src/bin/lfc/Makefile.am

@@ -18,7 +18,7 @@ CLEANFILES  = lfc_messages.h lfc_messages.cc s-messages
 
 man_MANS = kea-lfc.8
 DISTCLEANFILES = $(man_MANS)
-EXTRA_DIST = $(man_MANS) kea-lfc.xml
+EXTRA_DIST = $(man_MANS) kea-lfc.xml lfc.dox
 
 if GENERATE_DOCS
 kea-lfc.8: kea-lfc.xml

src/bin/lfc/kea-lfc.xml

@@ -62,10 +62,11 @@
     <title>DESCRIPTION</title>
     <para>
       The <command>kea-lfc</command> service process removes redundant
-      information for the files used to provide persistent storage for
+      information from the files used to provide persistent storage for
       the memfile data base backend.  The service is written to run as
-      a stand alone process.  While it can be started externally it
-      should be started by the Kea DHCP servers as desired and required.
+      a stand alone process.  While it can be started externally, there
+      is usually no need to do this.  It will be run on a periodic basis
+      by the Kea DHCP servers.
     </para>
   </refsect1>
 
@@ -129,9 +130,9 @@
         <listitem><para>
           PID file - When the <command>kea-lfc</command> process starts
           it attempts to determine if another instance of the process is
-          already running by examining the pid file.  If one is running
-          it aborts the new process.  If one isn't running it writes its
-          pid into the pid file.
+          already running by examining the pid file.  If one is already
+          running the new process is terminated.  If one isn't running it
+          writes its pid into the pid file.
         </para></listitem>
       </varlistentry>
 
@@ -150,7 +151,7 @@
         <term><option>-i</option></term>
         <listitem><para>
           Input or copy of lease file - Before the DHCP server invokes
-          <command>kea-lfc</command> it will copy the current lease file
+          <command>kea-lfc</command> it will move the current lease file
           here and then call <command>kea-lfc</command> with this file.
         </para></listitem>
       </varlistentry>

src/bin/lfc/lfc.dox

+// Copyright (C) 2015 Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+/**
+ @page lfc Lease File Cleanup Component
+
+The Lease File Cleanup component (kea-lfc) is a service process that removes redundant
+information from the files used to provide persistent storage for the memfile data base
+backend.
+
+When using the memfile database backend Kea servers store persistent lease information
+in lease files.  They add new lease information to the file by appending it to the end
+of the file.  In a typical lease file you might find several instances of the same lease,
+one for each time the client acquired or renewed the lease.  In order to remove this
+redundant information and to keep the lease files from growing without bound the
+Kea servers will periodically run kea-lfc.  This process will read the leases from
+the lease files and keep only the most recent instance of each lease.  When it finishes
+reading the leases it will write the unique leases to the output file.
+
+The design documentation for kea-lfc can be found here:
+<a href="http://kea.isc.org/wiki/LFCDesign"> LFC Design</a>
+
+While kea-lfc can be started externally, there is usually no need to do this as the
+Kea servers will run it on a periodic basis.
+
+@section lfcProcessing Processing
+kea-lfc operates on a set of files, using them for input and output
+of the lease entries and to indicate where it is in the process in case of an
+interruption.  Currently the caller must supply names for all of the files, in
+the future this requirement may be relaxed with the process getting the names
+from either the config file or from defaults.
+
+kea-lfc is built on the isc::lfc::LFCController class.  Effectively this provides
+a single external routine, isc::lfc::LFCController::launch, which is called with
+the arguments from the command line and proceeds to parse the arguments and
+then process the lease files.
+
+It uses isc::util::PIDFile to manipulate a PID file to mediate access to the
+leases.  When a new process is started it will check the PID file.  If the
+PID file exists and a process with that ID is still running already the new
+process will be terminated.  If no other process is running the PID of the
+new process is written to the file.
+
+It uses the isc::dhcp::LeaseFileLoader class to first read all of the leases
+into either isc::dhcp::Lease6Storage or isc::dhcp::Lease4Storage containers.
+The leases are read in the order they were written to the file and younger leases
+overwrite older leases for the same address.  When the process finishes reading
+the lease files there will be a single lease entry for each address used.  At
+this point the process again uses the isc::dhcp::LeaseFileLoader class to write
+an entry for each remaining lease into the output file.
+
+Lastly kea-lfc moves the files to indicate completion (see below) and removes
+the extra files then exits.
+
+@section lfcFiles File Manipulation
+
+This section is intended to provide a brief overview of how kea-lfc uses its
+files.  For a more in depth discussion of the design see
+<a href="http://kea.isc.org/wiki/LFCDesign"> LFC Design</a>
+
+There are four files used during the kea-lfc process: previous, input, output
+and finish.  They are used to both hold the leases and to indicate to the Kea
+servers and to other instances of kea-lfc what processing has been completed.
+
+The previous file is the result of a previous run of the kea-lfc process.
+
+The input (or copy) file is created by the kea server just before invoking
+the kea-lfc process.  The kea server will move it's current lease file to
+the input file and then start using a new current file.
+
+The output file is created by kea-lfc as it writes the leases to a file.  When
+kea-lfc finishes writing all the leases to the output file it renames the output
+file to be the finish file.  It then removes the previous and input files and
+renames the finish file to be the previous file.
+
+*/
+