Commit 653d1d85 authored by Mukund Sivaraman's avatar Mukund Sivaraman
Browse files

Merge branch 'master' into trac2236

Conflicts:
	src/bin/auth/auth_srv.cc
	src/bin/auth/tests/auth_srv_unittest.cc
parents 54ae641d c7123c28
495. [func] team
b10-auth now handles reconfiguration of data sources in
background using a separate thread. This means even if the new
configuration includes a large amount of data to be loaded into
memory (very large zones and/or a very large number of zones),
the reconfiguration doesn't block query handling.
(Multiple Trac tickets up to #2211)
494. [bug] jinmei
Fixed a problem that shutting down BIND 10 kept some of the
processes alive. It was two-fold: when the main bind10 process
......@@ -54,7 +62,7 @@
488. [build] jinmei
On configure, changed the search order for Python executable.
It first ties more specific file names such as "python3.2" before
It first tries more specific file names such as "python3.2" before
more generic "python3". This will prevent configure failure on
Mac OS X that installs Python3 via recent versions of Homebrew.
(Trac #2339, git 88db890d8d1c64de49be87f03c24a2021bcf63da)
......
......@@ -1126,6 +1126,7 @@ AC_CONFIG_FILES([Makefile
src/bin/bindctl/Makefile
src/bin/bindctl/tests/Makefile
src/bin/cfgmgr/Makefile
src/bin/cfgmgr/local_plugins/Makefile
src/bin/cfgmgr/plugins/Makefile
src/bin/cfgmgr/plugins/tests/Makefile
src/bin/cfgmgr/tests/Makefile
......
......@@ -55,6 +55,7 @@ b10_auth_SOURCES += auth_config.cc auth_config.h
b10_auth_SOURCES += command.cc command.h
b10_auth_SOURCES += common.h common.cc
b10_auth_SOURCES += statistics.cc statistics.h
b10_auth_SOURCES += datasrc_clients_mgr.h
b10_auth_SOURCES += datasrc_config.h datasrc_config.cc
b10_auth_SOURCES += main.cc
......
......@@ -21,6 +21,12 @@ namespace auth {
isc::log::Logger auth_logger("auth");
const int DBG_AUTH_START = DBGLVL_START_SHUT;
const int DBG_AUTH_SHUT = DBGLVL_START_SHUT;
const int DBG_AUTH_OPS = DBGLVL_COMMAND;
const int DBG_AUTH_DETAIL = DBGLVL_TRACE_BASIC;
const int DBG_AUTH_MESSAGES = DBGLVL_TRACE_DETAIL_DATA;
} // namespace auth
} // namespace isc
......@@ -28,21 +28,21 @@ namespace auth {
/// output.
// Debug messages indicating normal startup are logged at this debug level.
const int DBG_AUTH_START = DBGLVL_START_SHUT;
extern const int DBG_AUTH_START;
// Debug messages upon shutdown
const int DBG_AUTH_SHUT = DBGLVL_START_SHUT;
extern const int DBG_AUTH_SHUT;
// Debug level used to log setting information (such as configuration changes).
const int DBG_AUTH_OPS = DBGLVL_COMMAND;
extern const int DBG_AUTH_OPS;
// Trace detailed operations, including errors raised when processing invalid
// packets. (These are not logged at severities of WARN or higher for fear
// that a set of deliberately invalid packets set to the authoritative server
// could overwhelm the logging.)
const int DBG_AUTH_DETAIL = DBGLVL_TRACE_BASIC;
extern const int DBG_AUTH_DETAIL;
// This level is used to log the contents of packets received and sent.
const int DBG_AUTH_MESSAGES = DBGLVL_TRACE_DETAIL_DATA;
extern const int DBG_AUTH_MESSAGES;
/// Define the logger for the "auth" module part of b10-auth. We could define
/// a logger in each file, but we would want to define a common name to avoid
......
......@@ -57,6 +57,87 @@ At attempt to update the configuration the server with information
from the configuration database has failed, the reason being given in
the message.
% AUTH_DATASRC_CLIENTS_BUILDER_COMMAND data source builder received command: %1
A debug message, showing when the separate thread for maintaining data
source clients receives a command from the manager.
% AUTH_DATASRC_CLIENTS_BUILDER_FAILED data source builder thread stopped due to an exception: %1
The separate thread for maintaining data source clients has been
terminated due to some uncaught exception. When this happens, the
thread immediately terminates the entire process because the manager
cannot always catch this condition in a timely fashion and it would be
worse to keep running with such a half-broken state. This is really
an unexpected event and should generally indicate an internal bug.
It's advisable to file a bug report when this message is logged (and
b10-auth subsequently stops).
% AUTH_DATASRC_CLIENTS_BUILDER_FAILED_UNEXPECTED data source builder thread stopped due to an unexpected exception
This is similar to AUTH_DATASRC_CLIENTS_BUILDER_FAILED, but the
exception type indicates it's not thrown either within the BIND 10
implementation or other standard-compliant libraries. This may rather
indicate some run time failure than program errors. As in the other
failure case, the thread terminates the entire process immediately
after logging this message.
% AUTH_DATASRC_CLIENTS_BUILDER_RECONFIGURE_CONFIG_ERROR Error in data source configuration: %1
The thread for maintaining data source clients has received a command to
reconfigure, but the parameter data (the new configuration) contains an
error. The most likely cause is that the datasource-specific configuration
data is not what the data source expects. The system is still running with
the data sources that were previously configured (i.e. as if the
configuration has not changed), and the configuration data needs to be
checked.
The specific problem is printed in the log message.
% AUTH_DATASRC_CLIENTS_BUILDER_RECONFIGURE_DATASRC_ERROR Error setting up data source: %1
The thread for maintaining data source clients has received a command to
reconfigure, but a data source failed to set up. This may be a problem with
the data that is configured (e.g. unreadable files, inconsistent data,
parser problems, database connection problems, etc.), but it could be a bug
in the data source implementation as well. The system is still running with
the data sources that were previously configured (i.e. as if the
configuration has not changed).
The specific problem is printed in the log message.
% AUTH_DATASRC_CLIENTS_BUILDER_RECONFIGURE_ERROR Internal error setting up data source: %1
The thread for maintaining data source clients has received a command to
reconfigure, but raised an exception while setting up data sources. This is
most likely an internal error in a data source, or a bug in the data source
or the system itself, but it is probably a good idea to verify the
configuration first. The system is still running with the data sources that
were previously configured (i.e. as if the configuration has not changed).
The specific problem is printed in the log message.
% AUTH_DATASRC_CLIENTS_BUILDER_RECONFIGURE_STARTED data source reconfiguration started
The thread for maintaining data source clients has received a command to
reconfigure, and has now started this process.
% AUTH_DATASRC_CLIENTS_BUILDER_RECONFIGURE_SUCCESS data source reconfiguration completed succesfully
The thread for maintaining data source clients has finished reconfiguring
the data source clients, and is now running with the new configuration.
% AUTH_DATASRC_CLIENTS_BUILDER_STARTED data source builder thread started
A separate thread for maintaining data source clients has been started.
% AUTH_DATASRC_CLIENTS_BUILDER_STOPPED data source builder thread stopped
The separate thread for maintaining data source clients has been stopped.
% AUTH_DATASRC_CLIENTS_SHUTDOWN_ERROR error on waiting for data source builder thread: %1
This indicates that the separate thread for maintaining data source
clients had been terminated due to an uncaught exception, and the
manager notices that at its own termination. This is not an expected
event, because the thread is implemented so it catches all exceptions
internally. So, if this message is logged it's most likely some internal
bug, and it would be nice to file a bug report.
% AUTH_DATASRC_CLIENTS_SHUTDOWN_UNEXPECTED_ERROR Unexpected error on waiting for data source builder thread
Some exception happens while waiting for the termination of the
separate thread for maintaining data source clients. This shouldn't
happen in normal conditions; it should be either fatal system level
errors such as severe memory shortage or some internal bug. If that
happens, and if it's not in the middle of terminating b10-auth, it's
probably better to stop and restart it.
% AUTH_DATA_SOURCE data source database file: %1
This is a debug message produced by the authoritative server when it accesses a
datebase data source, listing the file that is being accessed.
......
......@@ -26,7 +26,6 @@
#include <exceptions/exceptions.h>
#include <util/buffer.h>
#include <util/threads/sync.h>
#include <dns/edns.h>
#include <dns/exceptions.h>
......@@ -53,6 +52,7 @@
#include <auth/query.h>
#include <auth/statistics.h>
#include <auth/auth_log.h>
#include <auth/datasrc_clients_mgr.h>
#include <boost/bind.hpp>
#include <boost/lexical_cast.hpp>
......@@ -268,26 +268,8 @@ public:
/// The TSIG keyring
const shared_ptr<TSIGKeyRing>* keyring_;
/// The data source client list
AuthSrv::DataSrcClientListsPtr datasrc_client_lists_;
shared_ptr<ConfigurableClientList> getDataSrcClientList(
const RRClass& rrclass)
{
#ifdef ENABLE_DEBUG
// Debug-build only check
if (!mutex_.locked()) {
isc_throw(isc::Unexpected, "Not locked!");
}
#endif
const std::map<RRClass, shared_ptr<ConfigurableClientList> >::
const_iterator it(datasrc_client_lists_->find(rrclass));
if (it == datasrc_client_lists_->end()) {
return (shared_ptr<ConfigurableClientList>());
} else {
return (it->second);
}
}
/// The data source client list manager
auth::DataSrcClientsMgr datasrc_clients_mgr_;
/// Bind the ModuleSpec object in config_session_ with
/// isc:config::ModuleSpec::validateStatistics.
......@@ -317,8 +299,6 @@ public:
isc::dns::Message& message,
bool done);
mutable util::thread::Mutex mutex_;
private:
bool xfrout_connected_;
AbstractXfroutClient& xfrout_client_;
......@@ -338,8 +318,6 @@ AuthSrvImpl::AuthSrvImpl(AbstractXfroutClient& xfrout_client,
xfrin_session_(NULL),
counters_(),
keyring_(NULL),
datasrc_client_lists_(new std::map<RRClass,
shared_ptr<ConfigurableClientList> >()),
ddns_base_forwarder_(ddns_forwarder),
ddns_forwarder_(NULL),
xfrout_connected_(false),
......@@ -490,6 +468,11 @@ AuthSrv::getIOService() {
return (impl_->io_service_);
}
isc::auth::DataSrcClientsMgr&
AuthSrv::getDataSrcClientsMgr() {
return (impl_->datasrc_clients_mgr_);
}
void
AuthSrv::setXfrinSession(AbstractSession* xfrin_session) {
impl_->xfrin_session_ = xfrin_session;
......@@ -648,15 +631,15 @@ AuthSrvImpl::processNormalQuery(const IOMessage& io_message, Message& message,
local_edns->setUDPSize(AuthSrvImpl::DEFAULT_LOCAL_UDPSIZE);
message.setEDNS(local_edns);
}
// Lock the client lists and keep them under the lock until the processing
// and rendering is done (this is the same mutex as from
// AuthSrv::getDataSrcClientListMutex()).
isc::util::thread::Mutex::Locker locker(mutex_);
// Get access to data source client list through the holder and keep the
// holder until the processing and rendering is done to avoid inter-thread
// race.
auth::DataSrcClientsMgr::Holder datasrc_holder(datasrc_clients_mgr_);
try {
const ConstQuestionPtr question = *message.beginQuestion();
const shared_ptr<datasrc::ClientList>
list(getDataSrcClientList(question->getClass()));
list(datasrc_holder.findClientList(question->getClass()));
if (list) {
const RRType& qtype = question->getType();
const Name& qname = question->getName();
......@@ -935,28 +918,6 @@ AuthSrv::destroyDDNSForwarder() {
}
}
AuthSrv::DataSrcClientListsPtr
AuthSrv::swapDataSrcClientLists(DataSrcClientListsPtr new_lists) {
#ifdef ENABLE_DEBUG
// Debug-build only check
if (!impl_->mutex_.locked()) {
isc_throw(isc::Unexpected, "Not locked!");
}
#endif
std::swap(new_lists, impl_->datasrc_client_lists_);
return (new_lists);
}
shared_ptr<ConfigurableClientList>
AuthSrv::getDataSrcClientList(const RRClass& rrclass) {
return (impl_->getDataSrcClientList(rrclass));
}
util::thread::Mutex&
AuthSrv::getDataSrcClientListMutex() const {
return (impl_->mutex_);
}
void
AuthSrv::setTCPRecvTimeout(size_t timeout) {
dnss_->setTCPRecvTimeout(timeout);
......
......@@ -19,6 +19,7 @@
#include <datasrc/factory.h>
#include <datasrc/client_list.h>
#include <datasrc/datasrc_config.h>
#include <dns/message.h>
#include <dns/opcode.h>
......@@ -34,7 +35,9 @@
#include <asiolink/asiolink.h>
#include <server_common/portconfig.h>
#include <auth/statistics.h>
#include <auth/datasrc_clients_mgr.h>
#include <boost/shared_ptr.hpp>
......@@ -193,6 +196,11 @@ public:
/// \brief Return pointer to the Checkin callback function
isc::asiolink::SimpleCallback* getCheckinProvider() const { return (checkin_); }
/// \brief Return data source clients manager.
///
/// \throw None
isc::auth::DataSrcClientsMgr& getDataSrcClientsMgr();
/// \brief Set the communication session with a separate process for
/// outgoing zone transfers.
///
......@@ -302,83 +310,10 @@ public:
/// If there was no forwarder yet, this method does nothing.
void destroyDDNSForwarder();
/// \brief Shortcut typedef used for swapDataSrcClientLists().
typedef boost::shared_ptr<std::map<
isc::dns::RRClass, boost::shared_ptr<
isc::datasrc::ConfigurableClientList> > >
DataSrcClientListsPtr;
/// \brief Swap the currently used set of data source client lists with
/// given one.
///
/// The "set" of lists is actually given in the form of map from
/// RRClasses to shared pointers to isc::datasrc::ConfigurableClientList.
///
/// This method returns the swapped set of lists, which was previously
/// used by the server.
///
/// This method is intended to be used by a separate method to update
/// the data source configuration "at once". The caller must hold
/// a lock for the mutex object returned by \c getDataSrcClientListMutex()
/// before calling this method.
///
/// The ownership of the returned pointer is transferred to the caller.
/// The caller is generally expected to release the resources used in
/// the old lists. Note that it could take longer time if some of the
/// data source clients contain a large size of in-memory data.
///
/// The caller can pass a NULL pointer. This effectively disables
/// any data source for the server.
///
/// \param new_lists Shared pointer to a new set of data source client
/// lists.
/// \return The previous set of lists. It can be NULL.
DataSrcClientListsPtr swapDataSrcClientLists(DataSrcClientListsPtr
new_lists);
/// \brief Returns the currently used client list for the class.
///
/// \param rrclass The class for which to get the list.
/// \return The list, or NULL if no list is set for the class.
boost::shared_ptr<isc::datasrc::ConfigurableClientList>
getDataSrcClientList(const isc::dns::RRClass& rrclass);
/// \brief Return a mutex for the client lists.
///
/// Background loading of data uses threads. Therefore we need to protect
/// the client lists by a mutex, so they don't change (or get destroyed)
/// during query processing. Get (and lock) this mutex whenever you do
/// something with the lists and keep it locked until you finish. This
/// is correct:
/// \code
/// {
/// Mutex::Locker locker(auth->getDataSrcClientListMutex());
/// boost::shared_ptr<isc::datasrc::ConfigurableClientList>
/// list(auth->getDataSrcClientList(RRClass::IN()));
/// // Do some processing here
/// }
/// \endcode
///
/// But this is not (it releases the mutex too soon):
/// \code
/// boost::shared_ptr<isc::datasrc::ConfigurableClientList> list;
/// {
/// Mutex::Locker locker(auth->getDataSrcClientListMutex());
/// list = auth->getDataSrcClientList(RRClass::IN()));
/// }
/// // Do some processing here
/// \endcode
///
/// \note This method is const even if you are allowed to modify
/// (lock) the mutex. It's because locking of the mutex is not really
/// a modification of the server object and it is needed to protect the
/// lists even on read-only operations.
isc::util::thread::Mutex& getDataSrcClientListMutex() const;
/// \brief Sets the timeout for incoming TCP connections
///
/// Incoming TCP connections that have not sent their data
/// withing this time are dropped.
/// within this time are dropped.
///
/// \param timeout The timeout (in milliseconds). If se to
/// zero, no timeouts are used, and the connection will remain
......
......@@ -18,7 +18,6 @@
#include <bench/benchmark_util.h>
#include <util/buffer.h>
#include <util/threads/sync.h>
#include <dns/message.h>
#include <dns/name.h>
......@@ -33,6 +32,7 @@
#include <auth/auth_srv.h>
#include <auth/auth_config.h>
#include <auth/datasrc_config.h>
#include <auth/datasrc_clients_mgr.h>
#include <auth/query.h>
#include <asiodns/asiodns.h>
......@@ -127,9 +127,9 @@ public:
OutputBuffer& buffer) :
QueryBenchMark(queries, query_message, buffer)
{
isc::util::thread::Mutex::Locker locker(
server_->getDataSrcClientListMutex());
server_->swapDataSrcClientLists(
// Note: setDataSrcClientLists() may be deprecated, but until then
// we use it because we want to be synchronized with the server.
server_->getDataSrcClientsMgr().setDataSrcClientLists(
configureDataSource(
Element::fromJSON("{\"IN\":"
" [{\"type\": \"sqlite3\","
......@@ -148,9 +148,7 @@ public:
OutputBuffer& buffer) :
QueryBenchMark(queries, query_message, buffer)
{
isc::util::thread::Mutex::Locker locker(
server_->getDataSrcClientListMutex());
server_->swapDataSrcClientLists(
server_->getDataSrcClientsMgr().setDataSrcClientLists(
configureDataSource(
Element::fromJSON("{\"IN\":"
" [{\"type\": \"MasterFiles\","
......
......@@ -15,13 +15,13 @@
#include <auth/command.h>
#include <auth/auth_log.h>
#include <auth/auth_srv.h>
#include <auth/datasrc_clients_mgr.h>
#include <cc/data.h>
#include <datasrc/client_list.h>
#include <config/ccsession.h>
#include <exceptions/exceptions.h>
#include <dns/rrclass.h>
#include <util/threads/sync.h>
#include <string>
......@@ -188,14 +188,11 @@ public:
if (!origin_elem) {
isc_throw(AuthCommandError, "Zone origin is missing");
}
Name origin(origin_elem->stringValue());
const Name origin(origin_elem->stringValue());
// We're going to work with the client lists. They may be used
// from a different thread too, protect them.
isc::util::thread::Mutex::Locker locker(
server.getDataSrcClientListMutex());
const boost::shared_ptr<isc::datasrc::ConfigurableClientList>
list(server.getDataSrcClientList(zone_class));
DataSrcClientsMgr::Holder holder(server.getDataSrcClientsMgr());
boost::shared_ptr<ConfigurableClientList> list =
holder.findClientList(zone_class);
if (!list) {
isc_throw(AuthCommandError, "There's no client list for "
......
// Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC")
//
// Permission to use, copy, modify, and/or distribute this software for any
// purpose with or without fee is hereby granted, provided that the above
// copyright notice and this permission notice appear in all copies.
//
// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
// AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
// PERFORMANCE OF THIS SOFTWARE.
#ifndef DATASRC_CLIENTS_MGR_H
#define DATASRC_CLIENTS_MGR_H 1
#include <util/threads/thread.h>
#include <util/threads/sync.h>
#include <log/logger_support.h>
#include <log/log_dbglevels.h>
#include <dns/rrclass.h>
#include <cc/data.h>
#include <datasrc/data_source.h>
#include <datasrc/client_list.h>
#include <auth/auth_log.h>
#include <auth/datasrc_config.h>
#include <boost/array.hpp>
#include <boost/bind.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/noncopyable.hpp>
#include <exception>
#include <list>
#include <utility>
namespace isc {
namespace auth {
namespace datasrc_clientmgr_internal {
// This namespace is essentially private for DataSrcClientsMgr(Base) and
// DataSrcClientsBuilder(Base). This is exposed in the public header
// only because these classes are templated (for testing purposes) and
// class internal has to be defined here.
/// \brief ID of commands from the DataSrcClientsMgr to DataSrcClientsBuilder.
enum CommandID {
NOOP, ///< Do nothing. Only useful for tests; no argument
RECONFIGURE, ///< Reconfigure the datasource client lists,
/// the argument to the command is the full new
/// datasources configuration.
SHUTDOWN, ///< Shutdown the builder; no argument
NUM_COMMANDS
};
/// \brief The data type passed from DataSrcClientsMgr to
/// DataSrcClientsBuilder.
///
/// The first element of the pair is the command ID, and the second element
/// is its argument. If the command doesn't take an argument it should be
/// a null pointer.
typedef std::pair<CommandID, data::ConstElementPtr> Command;
} // namespace datasrc_clientmgr_internal
/// \brief Frontend to the manager object for data source clients.
///
/// This class provides interfaces for configuring and updating a set of
/// data source clients "in the background". The user of this class can
/// assume any operation on this class can be done effectively non-blocking,
/// not suspending any delay-sensitive operations such as DNS query
/// processing. The only exception is the time when this class object
/// is destroyed (normally as a result of an implicit call to the destructor);
/// in the current implementation it can take time depending on what is
/// running "in the background" at the time of the call.
///
/// Internally, an object of this class invokes a separate thread to perform
/// time consuming operations such as loading large zone data into memory,
/// but such details are completely hidden from the user of this class.
///
/// This class is templated only so that we can test the class without
/// involving actual threads or mutex. Normal applications will only
/// need one specific specialization that has a typedef of
/// \c DataSrcClientsMgr.
template <typename ThreadType, typename BuilderType, typename MutexType,
typename CondVarType>
class DataSrcClientsMgrBase : boost::noncopyable {
private:
typedef std::map<dns::RRClass,
boost::shared_ptr<datasrc::ConfigurableClientList> >
ClientListsMap;
public:
/// \brief Thread-safe accessor to the data source client lists.
///
/// This class provides a simple wrapper for searching the client lists
/// stored in the DataSrcClientsMgr in a thread-safe manner.
/// It ensures the result of \c getClientList() can be used without
/// causing a race condition with other threads that can possibly use
/// the same manager throughout the lifetime of the holder object.
///
/// This also means the holder object is expected to have a short lifetime.
/// The application shouldn't try to keep it unnecessarily long.
/// It's normally expected to create the holder object on the stack
/// of a small scope and automatically let it be destroyed at the end
/// of the scope.
class Holder {
public:
Holder(DataSrcClientsMgrBase& mgr) :
mgr_(mgr), locker_(mgr_.map_mutex_)
{}
/// \brief Find a data source client list of a specified RR class.
///
/// It returns a pointer to the list stored in the manager if found,
/// otherwise it returns NULL. The manager keeps the ownership of
/// the pointed object. Also, it's not safe to get access to the
/// object beyond the scope of the holder object.
///
/// \note Since the ownership isn't transferred the return value
/// could be a bare pointer (and it's probably better in several
/// points). Unfortunately, some unit tests currently don't work
/// unless this method effectively shares the ownership with the
/// tests. That's the only reason why we return a shared pointer
/// for now. We should eventually fix it and change the return value
/// type (see Trac ticket #2395). Other applications must not
/// assume the ownership is actually shared.
boost::shared_ptr<datasrc::ConfigurableClientList> findClientList(
const dns::RRClass& rrclass)
{
const ClientListsMap::const_iterator
it = mgr_.clients_map_->find(rrclass);
if (it == mgr_.clients_map_->end()) {
return (boost::shared_ptr<datasrc::ConfigurableClientList>());
} else {
return (it->second);
}
}
private:
DataSrcClientsMgrBase& mgr_;
typename MutexType::Locker locker_;
};
/// \brief Constructor.
///
/// It internally invokes a separate thread and waits for further