Commit 6bc15852 authored by Andrei Pavel's avatar Andrei Pavel

Cassandra Host Data Source

Added src/share/database/scripts/cql/upgrade_1.0_to_2.0.sh to include host_reservations table in Cassandra.
Updated documentation to indicate that Cassandra now supports host reservations.
Added src/lib/dhcpsrv/cql_host_data_source.cc and cql_host_data_source.h.
Fixed a bug in CfgOption::mergeInternal() where formatted_value_ was not taken into consideration when merging, this is needed for merging the denormalized results on host retrieval in Cassandra. The method is not used elsewhere so there is no regression impact.
Added Cassandra support in HostDataSourceFactory.
Implemented a todo in mysql_host_data_source.cc
Added cql_host_data_source_unittest.cc
Functionality that Cassandra is not capable of like ORDER BY clause is
adjusted in generic_host_data_source_unittest.cc.
parent 50fd7903
......@@ -72,7 +72,7 @@
<listitem>
<simpara>
<command>lease-init</command> &mdash;
Initializes a new lease database. This is useful during a new
Initializes a new lease database. This is useful during a new
Kea installation. The database is initialized to the
latest version supported by the version of the software being
installed.
......@@ -160,12 +160,12 @@
<para>
<table frame="all" id="backends">
<title>List of available backends</title>
<tgroup cols='2'>
<tgroup cols='5'>
<colspec colname='feature'/>
<colspec colname='memfile'/>
<colspec colname='mysql'/>
<colspec colname='pgsql'/>
<colspec colname='cql'/>
<colspec colname='cql' colwidth='1.5*'/>
<thead>
<row>
<entry>Feature</entry>
......@@ -206,7 +206,7 @@
<entry>no</entry>
<entry>yes</entry>
<entry>yes</entry>
<entry>no</entry>
<entry>yes</entry>
</row>
<row>
......@@ -591,9 +591,8 @@ $ <userinput>kea-admin lease-upgrade pgsql -u <replaceable>database-user</replac
Cassandra, or Cassandra Query Language (CQL), is the newest backend
added to Kea. Since it was added recently and has not undergone as much
testing as other backends, it is considered experimental: please use
with caution. The CQL backend is currently able to store leases only. The
ability to store host reservations will likely be added some time in the
future.
with caution. The Casandra backend is currently able to store leases,
host reservations and options defined on a per host basis.
</para>
<para>
......
......@@ -2973,8 +2973,8 @@ It is merely echoed by the server
with classification using expressions.</para>
</section>
<section id="reservations4-mysql-pgsql">
<title>Storing Host Reservations in MySQL or PostgreSQL</title>
<section id="reservations4-mysql-pgsql-cql">
<title>Storing Host Reservations in MySQL, PostgreSQL or CQL (Cassandra)</title>
<para>
It is possible to store host reservations in MySQL or PostgreSQL database. See
......@@ -2990,13 +2990,6 @@ It is merely echoed by the server
arbitrarily set to 4096 bytes.</simpara></note>
</section>
<section id="reservations4-cql">
<title>Storing host reservations in CQL (Cassandra)</title>
<para>Kea currently does not support storing reservations in
Cassandra (CQL).</para>
</section>
<section id="reservations4-tuning">
<title>Fine Tuning DHCPv4 Host Reservation</title>
......
......@@ -144,6 +144,7 @@ endif
if HAVE_CQL
libkea_dhcpsrv_la_SOURCES += cql_connection.cc cql_connection.h
libkea_dhcpsrv_la_SOURCES += cql_exchange.cc cql_exchange.h
libkea_dhcpsrv_la_SOURCES += cql_host_data_source.cc cql_host_data_source.h
libkea_dhcpsrv_la_SOURCES += cql_lease_mgr.cc cql_lease_mgr.h
endif
......
......@@ -164,9 +164,9 @@ CfgOption::mergeInternal(const OptionSpaceContainer<OptionContainer,
// If there is no such option in the destination container,
// add one.
if (std::distance(range.first, range.second) == 0) {
dest_container.addItem(OptionDescriptor(src_opt->option_,
src_opt->persistent_),
*it);
dest_container.addItem(OptionDescriptor(
src_opt->option_, src_opt->persistent_,
src_opt->formatted_value_), *it);
}
}
}
......
......@@ -49,7 +49,7 @@ const uint32_t CQL_DRIVER_VERSION_MAJOR = CASS_VERSION_MAJOR;
const uint32_t CQL_DRIVER_VERSION_MINOR = CASS_VERSION_MINOR;
/// Define CQL schema version: 1.0
const uint32_t CQL_SCHEMA_VERSION_MAJOR = 1;
const uint32_t CQL_SCHEMA_VERSION_MAJOR = 2;
const uint32_t CQL_SCHEMA_VERSION_MINOR = 0;
/// @brief Common CQL connector pool
......
This diff is collapsed.
// Copyright (C) 2016 Deutsche Telekom AG.
//
// Author: Andrei Pavel <andrei.pavel@qualitance.com>
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#ifndef CQL_HOST_DATA_SOURCE_H
#define CQL_HOST_DATA_SOURCE_H
#include <dhcpsrv/base_host_data_source.h>
#include <dhcpsrv/cql_connection.h>
#include <stdint.h>
#include <string>
#include <utility>
namespace isc {
namespace dhcp {
/// Forward declaration to the implementation of @ref CqlHostDataSource.
class CqlHostDataSourceImpl;
/// @brief Cassandra host data source
///
/// Implements @ref isc::dhcp::BaseHostDataSource interface customized to
/// Cassandra. Use of this backend implies that a Cassandra database is
/// available and that the Kea schema has been created within it.
class CqlHostDataSource : public BaseHostDataSource {
public:
/// @brief Constructor
///
/// Uses the following keywords in the parameters passed to it to connect
/// to the database:
/// - name - Name of the database to which to connect (mandatory)
/// - host - Host to which to connect (optional, defaults to "localhost")
/// - user - Username under which to connect (optional)
/// - password - Password for "user" on the database (optional)
///
/// If the database is successfully opened, the version number in the
/// schema_version table will be checked against hard-coded value in the
/// implementation file.
///
/// Finally, all the CQL commands are pre-compiled.
///
/// @param parameters A data structure relating keywords and values
/// concerned with the database.
///
/// @throw isc::dhcp::NoDatabaseName Mandatory database name not given
/// @throw isc::dhcp::DbOpenError Error opening the database
/// @throw isc::dhcp::DbOperationError An operation on the open database has
/// failed.
explicit CqlHostDataSource(
const DatabaseConnection::ParameterMap& parameters);
/// @brief Virtual destructor.
///
/// Releases prepared CQL statements used by the backend.
virtual ~CqlHostDataSource();
/// @brief Adds a new @ref Host to the Cassandra database along with it's
/// reservations and options.
///
/// The implementations of this method should guard against duplicate
/// reservations for the same @ref Host, where possible. For example, when
/// the reservation for the same @ref HWAddr and @ref SubnetID is added
/// twice, @ref add() should throw a @ref DuplicateEntry exception. Note,
/// that usually it is impossible to guard against adding duplicated @ref
/// Host, where one instance is identified by @ref HWAddr, another one by
/// @ref DUID.
///
/// @param host pointer to the new @ref Host being added.
virtual void add(const HostPtr& host);
/// @brief Retrieves a single @ref Host connected to an IPv4 subnet.
///
/// Implementations of this method should guard against the case when
/// multiple instances of the @ref Host are present, e.g. when two @ref
/// Host objects are found, one for the @ref DUID, another one for the @ref
/// HWAddr. In such case, throw a @ref MultipleRecords exception.
///
/// @param subnet_id subnet identifier to filter by
/// @param hwaddr hardware address of the client to filter by or NULL if not
/// available
/// @param duid client identifier to filter by or NULL if not available
///
/// @return @ref ConstHostPtr to a @ref Host object using a specified @ref
/// HWAddr or @ref DUID
///
/// @throw BadValue if both or neither of subnet_id and duid are specified
virtual ConstHostPtr get4(const SubnetID& subnet_id,
const HWAddrPtr& hwaddr,
const DuidPtr& duid = DuidPtr()) const;
/// @brief Retrieves a @ref Host connected to an IPv4 subnet.
///
/// @param subnet_id subnet identifier to filter by
/// @param identifier_type identifier type to filter by
/// @param identifier_begin pointer to the beginning of a buffer containing a
/// host identifier to filter by
/// @param identifier_len length of the host identifier buffer
///
/// @return @ref Host object for which a reservation has been made using the
/// specified identifier
virtual ConstHostPtr get4(const SubnetID& subnet_id,
const Host::IdentifierType& identifier_type,
const uint8_t* identifier_begin,
const size_t identifier_len) const;
/// @brief Retrieves a @ref Host connected to an IPv4 subnet.
///
/// Note that dynamically allocated addresses and reserved addresses can
/// come into conflict. When the new address is assigned to a client, the
/// allocation mechanism should check if this address is not reserved for
/// some other @ref Host and not allocate it if a reservation is present.
///
/// @param subnet_id Subnet identifier.
/// @param address reserved IPv4 address.
///
/// @return Const @ref Host object
///
/// @throw BadValue if address in not a valid IPv4address
virtual ConstHostPtr get4(const SubnetID& subnet_id,
const asiolink::IOAddress& address) const;
/// @brief Retrieves a @ref Host connected to an IPv6 subnet.
///
/// Implementations of this method should guard against the case when
/// multiple instances of the @ref Host are present, e.g. when two
/// @ref Host objects are found, one for the @ref DUID, another one for the
/// @ref HWAddr. In such case, throw a @ref MultipleRecords exception.
///
/// @param subnet_id subnet identifier to filter by
/// @param hwaddr hardware address of the client to filter by or NULL if not
/// available
/// @param duid client identifier to filter by or NULL if not available
///
/// @return @ref Host object using a specified @ref HWAddr or @ref DUID
///
/// @throw BadValue if both or neither of subnet_id and duid are specified
virtual ConstHostPtr get6(const SubnetID& subnet_id,
const DuidPtr& duid,
const HWAddrPtr& hwaddr = HWAddrPtr()) const;
/// @brief Returns a @ref Host connected to an IPv6 subnet.
///
/// @param subnet_id subnet identifier to filter by
/// @param identifier_type identifier type to filter by
/// @param identifier_begin pointer to the beginning of a buffer containing a
/// host identifier to filter by
/// @param identifier_len length of the host identifier buffer
///
/// @return Const @ref Host object for which reservation has been made using
/// the specified identifier.
virtual ConstHostPtr get6(const SubnetID& subnet_id,
const Host::IdentifierType& identifier_type,
const uint8_t* identifier_begin,
const size_t identifier_len) const;
/// @brief Returns a @ref Host with the specified reservation prefix.
///
/// @param prefix IPv6 prefix for which the @ref Host object is searched.
/// @param prefix_len IPv6 prefix length.
///
/// @return Const @ref Host object using a specified HW address or DUID.
///
/// @throw MultipleRecords if two or more rows are returned from the
/// Cassandra database
virtual ConstHostPtr get6(const asiolink::IOAddress& prefix,
const uint8_t prefix_len) const;
/// @brief Returns a host connected to the IPv6 subnet and having
/// a reservation for a specified IPv6 address or prefix.
///
/// @param subnet_id Subnet identifier.
/// @param address reserved IPv6 address/prefix.
///
/// @return Const @c Host object using a specified IPv6 address/prefix.
virtual ConstHostPtr get6(const SubnetID& subnet_id,
const asiolink::IOAddress& address) const;
/// @brief Return all @ref Host objects for the specified @ref HWAddr or
/// @ref DUID.
///
/// Returns all @ref Host objects which represent reservations
/// for the specified HW address or DUID. Note, that this method may
/// return multiple reservations because a particular client may have
/// reservations in multiple subnets and the same client may be identified
/// by HW address or DUID. The server is unable to verify that the specific
/// DUID and HW address belong to the same client, until the client sends
/// a DHCP message.
///
/// Specifying both @ref HWAddr and @ref DUID is allowed for this method
/// and results in returning all objects that are associated with hardware
/// address OR duid. For example: if one @ref Host is associated with the
/// specified @ref HWAddr and another @ref Host is associated with the
/// specified @ref DUID, two hosts will be returned.
///
/// @param hwaddr HW address of the client or NULL if no HW address
/// available.
/// @param duid client id or NULL if not available, e.g. DHCPv4 client case.
///
/// @return collection of const @ref Host objects.
virtual ConstHostCollection getAll(const HWAddrPtr& hwaddr,
const DuidPtr& duid = DuidPtr()) const;
/// @brief Return all hosts connected to any subnet for which reservations
/// have been made using a specified identifier.
///
/// This method returns all @ref Host objects which represent reservations
/// for a specified identifier. This method may return multiple hosts
/// because a particular client may have reservations in multiple subnets.
///
/// @param identifier_type Identifier type.
/// @param identifier_begin Pointer to a beginning of a buffer containing
/// an identifier.
/// @param identifier_len Identifier length.
///
/// @return Collection of const @ref Host objects.
virtual ConstHostCollection
getAll(const Host::IdentifierType& identifier_type,
const uint8_t* identifier_begin,
const size_t identifier_len) const;
/// @brief Returns a collection of hosts using the specified IPv4 address.
///
/// This method may return multiple @ref Host objects if they are connected
/// to different subnets.
///
/// @param address IPv4 address for which the @ref Host object is searched.
///
/// @return Collection of const @ref Host objects.
virtual ConstHostCollection
getAll4(const asiolink::IOAddress& address) const;
/// @brief Returns description of the backend.
///
/// This description may be multiline text that describes the backend.
///
/// @return Description of the backend.
virtual std::string getDescription() const;
/// @brief Returns the name of the database.
///
/// @return database name
virtual std::string getName() const;
/// @brief Return backend type
///
/// @return backend type "cql"
virtual std::string getType() const;
/// @brief Retrieves schema version.
///
/// @return Version number stored in the database, as a pair of unsigned
/// integers. "first" is the major version number, "second" is the
/// minor version number.
///
/// @throw isc::dhcp::DbOperationError An operation on the open database
/// has failed.
virtual std::pair<uint32_t, uint32_t> getVersion() const;
/// @brief Commit Transactions
///
/// Commits all pending database operations.
virtual void commit();
/// @brief Rollback Transactions
///
/// Rolls back all pending database operations.
virtual void rollback();
private:
/// @brief Pointer to the implementation of the @ref CqlHostDataSource.
CqlHostDataSourceImpl* impl_;
}; // class CqlHostDataSource
}; // namespace dhcp
}; // namespace isc
#endif // CQL_HOST_DATA_SOURCE_H
......@@ -175,9 +175,6 @@ with the specified address to the Cassandra backend database.
% DHCPSRV_CQL_COMMIT committing to Cassandra database
A commit call been issued on the server. For Cassandra, this is a no-op.
% DHCPSRV_CQL_BEGIN_TRANSACTION committing to Cassandra database.
The server has issued a begin transaction call.
% DHCPSRV_CQL_DB opening Cassandra lease database: %1
This informational message is logged when a DHCP server (either V4 or
V6) is about to open a Cassandra lease database. The parameters of
......@@ -264,6 +261,10 @@ subnet ID and hardware address.
A debug message issued when the server is about to obtain schema version
information from the Cassandra database.
% DHCPSRV_CQL_HOST_RETRIEVE_ERROR Unable to retrieve host reservation or option: %1
A host couldn't be fully retrieved. An exception was thrown, but the server
should continue running.
% DHCPSRV_CQL_ROLLBACK rolling back Cassandra database
The code has issued a rollback call. For Cassandra, this is
a no-op.
......@@ -276,6 +277,30 @@ lease from the Cassandra database for the specified address.
A debug message issued when the server is attempting to update IPv6
lease from the Cassandra database for the specified address.
% DHCPSRV_CQL_HOST_DB Connecting to CQL hosts database: %1
An informational message logged when the CQL hosts database is about to be
connected to. The parameters of the connection including database name and
username needed to access it (but not the password if any) are logged.
% DHCPSRV_CQL_HOST_DB_GET_VERSION obtaining schema version information for the CQL hosts database
A debug message issued when the server is about to obtain schema version
information from the CQL hosts database.
% DHCPSRV_CQL_HOST_ADD Adding host information to the database
An informational message logged when options belonging to any reservation from a
single host are inserted.
% DHCPSRV_CQL_HOST_GET_ALL Retrieving multiple hosts from a CQL database
An informational message logged when multiple hosts from a CQL database are retrieved.
% DHCPSRV_CQL_HOST_GET4 Retrieving one DHCPv4 host from a CQL database
An informational message logged when a DHCP server is about to retrieve one
host from a CQL database by IPv4 criteria.
% DHCPSRV_CQL_HOST_GET6 Retrieving one DHCPv6 host from a CQL database
An informational message logged when a DHCP server is about to retrieve one
host from a CQL database by IPv6 criteria.
% DHCPSRV_DHCP4O6_RECEIVED_BAD_PACKET received bad DHCPv4o6 packet: %1
A bad DHCPv4o6 packet was received.
......
......@@ -22,7 +22,7 @@
namespace isc {
namespace dhcp {
/// @brief HostID (used only when storing in MySQL or Postgres)
/// @brief HostID (used only when storing in MySQL, PostgreSQL or Cassandra)
typedef uint64_t HostID;
/// @brief IPv6 reservation for a host.
......@@ -531,13 +531,13 @@ public:
/// @brief Returns information about the host in the textual format.
std::string toText() const;
/// @brief Sets Host ID (primary key in MySQL and Postgres backends)
/// @brief Sets Host ID (primary key in MySQL, PostgreSQL and Cassandra backends)
/// @param id HostId value
void setHostId(HostID id) {
host_id_ = id;
}
/// @brief Returns Host ID (primary key in MySQL and Postgres backends)
/// @brief Returns Host ID (primary key in MySQL, PostgreSQL and Cassandra backends)
/// @return id HostId value (or 0 if not set)
HostID getHostId() const {
return (host_id_);
......@@ -584,7 +584,7 @@ private:
std::string boot_file_name_;
/// @brief HostID (a unique identifier assigned when the host is stored in
/// MySQL or Pgsql)
/// MySQL, PostgreSQL or Cassandra)
uint64_t host_id_;
/// @brief Pointer to the DHCPv4 option data configuration for this host.
......
......@@ -18,6 +18,10 @@
#include <dhcpsrv/pgsql_host_data_source.h>
#endif
#ifdef HAVE_CQL
#include <dhcpsrv/cql_host_data_source.h>
#endif
#include <boost/algorithm/string.hpp>
#include <boost/foreach.hpp>
#include <boost/scoped_ptr.hpp>
......@@ -76,8 +80,10 @@ HostDataSourceFactory::create(const std::string& dbaccess) {
#ifdef HAVE_CQL
if (db_type == "cql") {
isc_throw(NotImplemented, "Sorry, CQL backend for host reservations "
"is not implemented yet.");
LOG_INFO(dhcpsrv_logger, DHCPSRV_CQL_HOST_DB)
.arg(DatabaseConnection::redactedAccessString(parameters));
getHostDataSourcePtr().reset(new CqlHostDataSource(parameters));
return;
}
#endif
......
......@@ -2519,7 +2519,11 @@ MySqlHostDataSource::get4(const SubnetID& subnet_id,
ConstHostPtr
MySqlHostDataSource::get4(const SubnetID& subnet_id,
const asiolink::IOAddress& address) const {
/// @todo: check that address is really v4, not v6.
// Check that address is IPv4, not IPv6.
if (!address.isV4()) {
isc_throw(BadValue, "MySqlHostDataSource::get4(2): wrong address type, "
"address supplied is not an IPv4 address");
}
// Set up the WHERE clause value
MYSQL_BIND inbind[2];
......
......@@ -119,6 +119,7 @@ libdhcpsrv_unittests_SOURCES += pgsql_lease_mgr_unittest.cc
endif
if HAVE_CQL
libdhcpsrv_unittests_SOURCES += cql_lease_mgr_unittest.cc
libdhcpsrv_unittests_SOURCES += cql_host_data_source_unittest.cc
endif
libdhcpsrv_unittests_SOURCES += pool_unittest.cc
libdhcpsrv_unittests_SOURCES += srv_config_unittest.cc
......
// Copyright (C) 2015-2016 Internet Systems Consortium, Inc. ("ISC")
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
#include <config.h>
#include <asiolink/io_address.h>
#include <dhcpsrv/cql_connection.h>
#include <dhcpsrv/cql_host_data_source.h>
#include <dhcpsrv/cql_lease_mgr.h>
#include <dhcpsrv/host.h>
#include <dhcpsrv/host_data_source_factory.h>
#include <dhcpsrv/tests/generic_host_data_source_unittest.h>
#include <dhcpsrv/tests/test_utils.h>
#include <dhcpsrv/testutils/cql_schema.h>
#include <exceptions/exceptions.h>
#include <gtest/gtest.h>
#include <algorithm>
#include <iostream>
#include <sstream>
#include <string>
#include <utility>
using namespace isc;
using namespace isc::asiolink;
using namespace isc::dhcp;
using namespace isc::dhcp::test;
using namespace std;
namespace {
class CqlHostDataSourceTest : public GenericHostDataSourceTest {
public:
/// @brief Constructor
///
/// Deletes everything from the database and opens it.
CqlHostDataSourceTest() {
// Ensure schema is the correct one.
destroyCqlSchema(false, true);
createCqlSchema(false, true);
// Connect to the database
try {
HostDataSourceFactory::create(validCqlConnectionString());
} catch (...) {
std::cerr << "*** ERROR: unable to open database. The test"
"*** environment is broken and must be fixed before"
"*** the CQL tests will run correctly."
"*** The reason for the problem is described in the"
"*** accompanying exception output.";
throw;
}
hdsptr_ = HostDataSourceFactory::getHostDataSourcePtr();
}
/// @brief Destructor
///
/// Rolls back all pending transactions. The deletion of myhdsptr_ will
/// close the database. Then reopen it and delete everything created by the
/// test.
virtual ~CqlHostDataSourceTest() {
hdsptr_->rollback();
HostDataSourceFactory::destroy();
destroyCqlSchema(false, true);
}
/// @brief Reopen the database
///
/// Closes the database and re-open it. Anything committed should be
/// visible.
///
/// Parameter is ignored for CQL backend as the v4 and v6 leases share
/// the same database.
void reopen(Universe) {
HostDataSourceFactory::destroy();
HostDataSourceFactory::create(validCqlConnectionString());
hdsptr_ = HostDataSourceFactory::getHostDataSourcePtr();
}
};
/// @brief Check that database can be opened
///
/// This test checks if the CqlHostDataSource can be instantiated. This happens
/// only if the database can be opened. Note that this is not part of the
/// CqlLeaseMgr test fixure set. This test checks that the database can be
/// opened: the fixtures assume that and check basic operations.
TEST(CqlHostDataSource, OpenDatabase) {
// Schema needs to be created for the test to work.
destroyCqlSchema(false, true);
createCqlSchema(false, true);
// Check that lease manager open the database opens correctly and tidy up.
// If it fails, print the error message.
try {
HostDataSourceFactory::create(validCqlConnectionString());
EXPECT_NO_THROW((void)HostDataSourceFactory::getHostDataSourcePtr());
HostDataSourceFactory::destroy();
} catch (const isc::Exception& ex) {
FAIL() << "*** ERROR: unable to open database, reason:\n"
<< " " << ex.what() << "\n"
<< "*** The test environment is broken and must be fixed\n"
<< "*** before the CQL tests will run correctly.\n";
}
// Check that lease manager open the database opens correctly with a longer
// timeout. If it fails, print the error message.
try {
std::string connection_string = validCqlConnectionString() +
std::string(" ") +
std::string(VALID_TIMEOUT);
HostDataSourceFactory::create(connection_string);
EXPECT_NO_THROW((void)HostDataSourceFactory::getHostDataSourcePtr());
HostDataSourceFactory::destroy();
} catch (const isc::Exception& ex) {
FAIL() << "*** ERROR: unable to open database, reason:\n"