Commit 23070d21 authored by Naoki Kambe's avatar Naoki Kambe
Browse files

[2158] Merge branch 'master' of ssh://git.bind10.isc.org/var/bind10/git/bind10 into trac2158

Conflicts:
	ChangeLog
parents 9c938d15 f2f058d5
nnn. [func] naokikambe
Added Xfrout statistics counters: notifyoutv4, notifyoutv6, xfrrej, and
xfrreqdone. These are per-zone type counters. The value of these
counters can be seen with zone name by invoking "Stats show Xfrout" via
bindctl.
(Trac #2158, git TBD)
nnn. [func] naokikambe
The stats module was supported differential statistics updates. Each
module can return only statistics data which are updated since the last
time it sent to the stats module. The purpose of differential updating
is reducing amounts of statistics data sent in the message queue.
(Trac #2179, git TBD)
nnn. [func] naokikambe
Added Xfrout statistics counters: notifyoutv4, notifyoutv6, xfrrej, and
xfrreqdone. These are per-zone type counters. The value of these
counters can be seen with zone name by invoking "Stats show Xfrout" via
bindctl.
(Trac #2158, git TBD)
470. [func] naokikambe
The stats module now supports partial statistics updates. Each
module can return only statistics data which have been updated since
the last time it sent them to the stats module. The purpose of partial
updates is to reduce the amount of statistics data sent through the
message queue.
(Trac #2179, git d659abdd9f3f369a29830831297f64484ac7b051)
469. [bug] jelte
libdatasrc: the data source client list class now ignores zone
content problems (such as out-of-zone data) in MasterFiles type
zones, instead of aborting the entire configuration. It only logs
an error, and all other zones and datasources are still loaded. The
error log message has been improved to include the zone origin and
source file name. As a result of this change, b10-auth no longer
exits upon encountering such errors during startup.
(Trac #2178, git a75ed413e8a1c8e3702beea4811a46a1bf519bbd)
468. [func]* naokikambe, fujiwara
b10-stats polls the bind10 and b10-auth with new 'getstats' command
to retrieve statistics data. The "poll-interval" parameter in
b10-stats is for configuring the polling interval. All statistics
data collected once are preserved while b10-stats is running.
The "sendstats" command was removed from bind10 and b10-auth. The
"statistics-interval" configuration item was removed from b10-auth.
(Trac #2136, git dcb5ce50b4b4e50d28247d5f8b5cb8d90bda942a)
(Trac #2137, git d53bb65a43f6027b15a6edc08c137951e3ce5e0e)
(Trac #2138, git b34e3313460eebc9c272ca8c1beb27297c195150)
bind10-devel-20120816 released on August 16, 2012
......@@ -144,7 +166,7 @@ bind10-devel-20120816 released on August 16, 2012
is 'Dhcp4 shutdown'.
(Trac #1651, git 7e16a5a50d3311e63d10a224ec6ebcab5f25f62c)
bind10-devel-20120621 released on June 21. 2012
bind10-devel-20120621 released on June 21, 2012
449. [bug] muks
b10-xfin: fixed a bug where xfrin sent the wrong notification
......@@ -205,7 +227,7 @@ bind10-devel-20120621 released on June 21. 2012
file) was removed.
(Trac #1281, git 900fc8b420789a8c636bcf20fdaffc60bc1041e0)
bind10-devel-20120517 released on May 17. 2012
bind10-devel-20120517 released on May 17, 2012
440. [func] muks
bindctl: improved some error messages so they will be more
......
......@@ -1166,6 +1166,7 @@ AC_CONFIG_FILES([Makefile
src/lib/datasrc/Makefile
src/lib/datasrc/memory/Makefile
src/lib/datasrc/memory/tests/Makefile
src/lib/datasrc/memory/benchmarks/Makefile
src/lib/datasrc/tests/Makefile
src/lib/datasrc/tests/testdata/Makefile
src/lib/xfr/Makefile
......
......@@ -1633,21 +1633,6 @@ can use various data source backends.
</simpara>
</listitem>
</varlistentry>
<varlistentry>
<term>statistics-interval</term>
<listitem>
<simpara>
<varname>statistics-interval</varname> is the timer interval
in seconds for <command>b10-auth</command> to share its
statistics information to
<citerefentry><refentrytitle>b10-stats</refentrytitle><manvolnum>8</manvolnum></citerefentry>.
Statistics updates can be disabled by setting this to 0.
The default is 60.
</simpara>
</listitem>
</varlistentry>
</variablelist>
</para>
......@@ -1680,13 +1665,14 @@ can use various data source backends.
</varlistentry>
<varlistentry>
<term>sendstats</term>
<term>getstats</term>
<listitem>
<simpara>
<command>sendstats</command> tells <command>b10-auth</command>
<command>getstats</command> requests <command>b10-auth</command>
to send its statistics data to
<citerefentry><refentrytitle>b10-stats</refentrytitle><manvolnum>8</manvolnum></citerefentry>
immediately.
<citerefentry><refentrytitle>b10-stats</refentrytitle>
<manvolnum>8</manvolnum></citerefentry>
as a response of the command.
</simpara>
</listitem>
</varlistentry>
......
......@@ -420,16 +420,6 @@ This is a debug message reporting that an incoming NOTIFY was received.
</para></listitem>
</varlistentry>
<varlistentry id="AUTH_RECEIVED_SENDSTATS">
<term>AUTH_RECEIVED_SENDSTATS command 'sendstats' received</term>
<listitem><para>
This is a debug message issued when the authoritative server has received
a command from the statistics module to send it data. The 'sendstats'
command is handled differently to other commands, which is why the debug
message associated with it has its own code.
</para></listitem>
</varlistentry>
<varlistentry id="AUTH_RESPONSE_FAILURE">
<term>AUTH_RESPONSE_FAILURE exception while building response to query: %1</term>
<listitem><para>
......
......@@ -56,11 +56,6 @@
}]
}
},
{ "item_name": "statistics-interval",
"item_type": "integer",
"item_optional": true,
"item_default": 60
},
{
"item_name": "listen_on",
"item_type": "list",
......@@ -110,8 +105,8 @@
]
},
{
"command_name": "sendstats",
"command_description": "Send data to a statistics module at once",
"command_name": "getstats",
"command_description": "Retrieve statistics data",
"command_args": []
},
{
......
......@@ -53,37 +53,6 @@ public:
virtual void commit() {};
};
/// A derived \c AuthConfigParser class for the "statistics-internal"
/// configuration identifier.
class StatisticsIntervalConfig : public AuthConfigParser {
public:
StatisticsIntervalConfig(AuthSrv& server) :
server_(server), interval_(0)
{}
virtual void build(ConstElementPtr config_value) {
const int32_t config_interval = config_value->intValue();
if (config_interval < 0) {
isc_throw(AuthConfigError, "Negative statistics interval value: "
<< config_interval);
}
if (config_interval > 86400) {
isc_throw(AuthConfigError, "Statistics interval value "
<< config_interval
<< " must be equal to or shorter than 86400");
}
interval_ = config_interval;
}
virtual void commit() {
// setStatisticsTimerInterval() is not 100% exception free. But
// exceptions should happen only in a very rare situation, so we
// let them be thrown and subsequently regard them as a fatal error.
server_.setStatisticsTimerInterval(interval_);
}
private:
AuthSrv& server_;
uint32_t interval_;
};
/// A special parser for testing: it throws from commit() despite the
/// suggested convention of the class interface.
class ThrowerCommitConfig : public AuthConfigParser {
......@@ -155,9 +124,7 @@ createAuthConfigParser(AuthSrv& server, const std::string& config_id) {
// simplicity. In future we'll probably generalize it using map-like
// data structure, and may even provide external register interface so
// that it can be dynamically customized.
if (config_id == "statistics-interval") {
return (new StatisticsIntervalConfig(server));
} else if (config_id == "listen_on") {
if (config_id == "listen_on") {
return (new ListenAddressConfig(server));
} else if (config_id == "_commit_throw") {
// This is for testing purpose only and should not appear in the
......
......@@ -122,11 +122,6 @@ a NOTIFY packet that an RR type of something other than SOA in the
question section. (The RR type received is included in the message.) The
server will return a FORMERR error to the sender.
% AUTH_NO_STATS_SESSION session interface for statistics is not available
The authoritative server had no session with the statistics module at the
time it attempted to send it data: the attempt has been abandoned. This
could be an error in configuration.
% AUTH_NO_XFRIN received NOTIFY but XFRIN session is not running
This is a debug message produced by the authoritative server when it receives
a NOTIFY packet but the XFRIN process is not running. The packet will be
......@@ -169,12 +164,6 @@ a command on the command channel.
% AUTH_RECEIVED_NOTIFY received incoming NOTIFY for zone name %1, zone class %2
This is a debug message reporting that an incoming NOTIFY was received.
% AUTH_RECEIVED_SENDSTATS command 'sendstats' received
This is a debug message issued when the authoritative server has received
a command from the statistics module to send it data. The 'sendstats'
command is handled differently to other commands, which is why the debug
message associated with it has its own code.
% AUTH_RESPONSE_FAILURE exception while building response to query: %1
This is a debug message, generated by the authoritative server when an
attempt to create a response to a received DNS packet has failed. The
......@@ -203,7 +192,7 @@ multiple lines of text. For this reason, it is suggested that this log message
not be routed to the syslog file, where the multiple lines could confuse
programs that expect a format of one message per line.
% AUTH_SEND_NORMAL_RESPONSE sending an error response (%1 bytes):\n%2
% AUTH_SEND_NORMAL_RESPONSE sending a normal response (%1 bytes):\n%2
This is a debug message recording that the authoritative server is sending
a response to the originator of a query.
......@@ -242,37 +231,6 @@ is not running, b10-auth will respond to UPDATE requests with rcode NOTIMP.
When b10-ddns is running, b10-ddns will handle and respond to the UPDATE
message.
% AUTH_STATS_CHANNEL_CREATED STATS session channel created
This is a debug message indicating that the authoritative server has
created a channel to the statistics process. It is issued during server
startup is an indication that the initialization is proceeding normally.
% AUTH_STATS_CHANNEL_ESTABLISHED STATS session channel established
This is a debug message indicating that the authoritative server
has established communication over the previously created statistics
channel. It is issued during server startup is an indication that the
initialization is proceeding normally.
% AUTH_STATS_COMMS communication error in sending statistics data: %1
An error was encountered when the authoritative server tried to send data
to the statistics daemon. The message includes additional information
describing the reason for the failure.
% AUTH_STATS_TIMEOUT timeout while sending statistics data: %1
The authoritative server sent data to the statistics daemon but received
no acknowledgement within the specified time. The message includes
additional information describing the reason for the failure.
% AUTH_STATS_TIMER_DISABLED statistics timer has been disabled
This is a debug message indicating that the statistics timer has been
disabled in the authoritative server and no statistics information is
being produced.
% AUTH_STATS_TIMER_SET statistics timer set to %1 second(s)
This is a debug message indicating that the statistics timer has been
enabled and that the authoritative server will produce statistics data
at the specified interval.
% AUTH_STOP_DDNS_FORWARDER DDNS UPDATE handling stopped
This is a debug message indicating that b10-auth has received a message
that it should stop internally forwarding UPDATE message to b10-ddns.
......
......@@ -247,9 +247,6 @@ public:
ModuleCCSession* config_session_;
AbstractSession* xfrin_session_;
/// Interval timer for periodic submission of statistics counters.
IntervalTimer statistics_timer_;
/// Query counters for statistics
AuthCounters counters_;
......@@ -320,7 +317,6 @@ AuthSrvImpl::AuthSrvImpl(AbstractXfroutClient& xfrout_client,
BaseSocketSessionForwarder& ddns_forwarder) :
config_session_(NULL),
xfrin_session_(NULL),
statistics_timer_(io_service_),
counters_(),
keyring_(NULL),
ddns_base_forwarder_(ddns_forwarder),
......@@ -479,43 +475,11 @@ AuthSrv::setConfigSession(ModuleCCSession* config_session) {
impl_->registerStatisticsValidator();
}
void
AuthSrv::setStatisticsSession(AbstractSession* statistics_session) {
impl_->counters_.setStatisticsSession(statistics_session);
}
ModuleCCSession*
AuthSrv::getConfigSession() const {
return (impl_->config_session_);
}
uint32_t
AuthSrv::getStatisticsTimerInterval() const {
return (impl_->statistics_timer_.getInterval() / 1000);
}
void
AuthSrv::setStatisticsTimerInterval(uint32_t interval) {
if (interval == impl_->statistics_timer_.getInterval()) {
return;
}
if (interval > 86400) {
// It can't occur since the value is checked in
// statisticsIntervalConfig::build().
isc_throw(InvalidParameter, "Too long interval: " << interval);
}
if (interval == 0) {
impl_->statistics_timer_.cancel();
LOG_DEBUG(auth_logger, DBG_AUTH_OPS, AUTH_STATS_TIMER_DISABLED);
} else {
impl_->statistics_timer_.setup(boost::bind(&AuthSrv::submitStatistics,
this),
interval * 1000);
LOG_DEBUG(auth_logger, DBG_AUTH_OPS, AUTH_STATS_TIMER_SET)
.arg(interval);
}
}
void
AuthSrv::processMessage(const IOMessage& io_message, Message& message,
OutputBuffer& buffer, DNSServer* server)
......@@ -882,8 +846,8 @@ AuthSrv::updateConfig(ConstElementPtr new_config) {
}
}
bool AuthSrv::submitStatistics() const {
return (impl_->counters_.submitStatistics());
ConstElementPtr AuthSrv::getStatistics() const {
return (impl_->counters_.getStatistics());
}
uint64_t
......
......@@ -17,7 +17,6 @@
#include <string>
#include <cc/data.h>
#include <config/ccsession.h>
#include <datasrc/factory.h>
#include <dns/message.h>
......@@ -203,64 +202,21 @@ public:
///
void setXfrinSession(isc::cc::AbstractSession* xfrin_session);
/// \brief Set the communication session with Statistics.
///
/// This function never throws an exception as far as
/// AuthCounters::setStatisticsSession() doesn't throw.
///
/// Note: this interface is tentative. We'll revisit the ASIO and
/// session frameworks, at which point the session will probably
/// be passed on construction of the server.
///
/// \param statistics_session A Session object over which statistics
/// information is exchanged with statistics module.
/// The session must be established before setting in the server
/// object.
/// Ownership isn't transferred: the caller is responsible for keeping
/// this object to be valid while the server object is working and for
/// disconnecting the session and destroying the object when the server
/// is shutdown.
void setStatisticsSession(isc::cc::AbstractSession* statistics_session);
/// Return the interval of periodic submission of statistics in seconds.
///
/// If the statistics submission is disabled, it returns 0.
///
/// This method never throws an exception.
uint32_t getStatisticsTimerInterval() const;
/// Set the interval of periodic submission of statistics.
///
/// If the specified value is non 0, the \c AuthSrv object will submit
/// its statistics to the statistics module every \c interval seconds.
/// If it's 0, and \c AuthSrv currently submits statistics, the submission
/// will be disabled. \c interval must be equal to or shorter than 86400
/// seconds (1 day).
///
/// This method should normally not throw an exception; however, its
/// underlying library routines may involve resource allocation, and
/// when it fails it would result in a corresponding standard exception.
///
/// \param interval The submission interval in seconds if non 0;
/// or a value of 0 to disable the submission.
void setStatisticsTimerInterval(uint32_t interval);
/// \brief Submit statistics counters to statistics module.
/// \brief Returns statistics data
///
/// This function can throw an exception from
/// AuthCounters::submitStatistics().
/// AuthCounters::getStatistics().
///
/// \return true on success, false on failure (e.g. session timeout,
/// session error).
bool submitStatistics() const;
/// \return JSON format statistics data.
isc::data::ConstElementPtr getStatistics() const;
/// \brief Get the value of counter in the AuthCounters.
///
/// This function calls AuthCounters::getCounter() and
/// This function calls AuthCounters::getStatistics() and
/// returns its return value.
///
/// This function never throws an exception as far as
/// AuthCounters::getCounter() doesn't throw.
/// AuthCounters::getStatistics() doesn't throw.
///
/// Note: Currently this function is for testing purpose only.
///
......
......@@ -14,13 +14,6 @@
#include <config.h>
#include <stdlib.h>
#include <iostream>
#include <vector>
#include <boost/shared_ptr.hpp>
#include <bench/benchmark.h>
#include <bench/benchmark_util.h>
......@@ -37,11 +30,19 @@
#include <auth/auth_srv.h>
#include <auth/auth_config.h>
#include <auth/datasrc_configurator.h>
#include <auth/query.h>
#include <asiodns/asiodns.h>
#include <asiolink/asiolink.h>
#include <boost/shared_ptr.hpp>
#include <stdlib.h>
#include <iostream>
#include <vector>
using namespace std;
using namespace isc;
using namespace isc::data;
......@@ -124,8 +125,13 @@ public:
OutputBuffer& buffer) :
QueryBenchMark(queries, query_message, buffer)
{
server_->updateConfig(Element::fromJSON("{\"database_file\": \"" +
string(datasrc_file) + "\"}"));
DataSourceConfigurator::testReconfigure(
server_.get(),
Element::fromJSON("{\"IN\":"
" [{\"type\": \"sqlite3\","
" \"params\": {"
" \"database_file\": \"" +
string(datasrc_file) + "\"}}]}"));
}
};
......@@ -138,14 +144,14 @@ public:
OutputBuffer& buffer) :
QueryBenchMark(queries, query_message, buffer)
{
configureAuthServer(*server_,
Element::fromJSON(
"{\"datasources\": "
" [{\"type\": \"memory\","
" \"zones\": [{\"origin\": \"" +
string(zone_origin) + "\","
" \"file\": \"" +
string(zone_file) + "\"}]}]}"));
DataSourceConfigurator::testReconfigure(
server_.get(),
Element::fromJSON("{\"IN\":"
" [{\"type\": \"MasterFiles\","
" \"cache-enable\": true, "
" \"params\": {\"" +
string(zone_origin) + "\": \"" +
string(zone_file) + "\"}}]}"));
}
};
......@@ -187,8 +193,9 @@ enum DataSrcType {
void
usage() {
cerr <<
"Usage: query_bench [-n iterations] [-t datasrc_type] [-o origin] "
"datasrc_file query_datafile\n"
"Usage: query_bench [-d] [-n iterations] [-t datasrc_type] [-o origin]"
" datasrc_file query_datafile\n"
" -d Enable debug logging to stdout\n"
" -n Number of iterations per test case (default: "
<< ITERATION_DEFAULT << ")\n"
" -t Type of data source: sqlite3|memory (default: sqlite3)\n"
......@@ -208,7 +215,8 @@ main(int argc, char* argv[]) {
int iteration = ITERATION_DEFAULT;
const char* opt_datasrc_type = "sqlite3";
const char* origin = NULL;
while ((ch = getopt(argc, argv, "n:t:o:")) != -1) {
bool debug_log = false;
while ((ch = getopt(argc, argv, "dn:t:o:")) != -1) {
switch (ch) {
case 'n':
iteration = atoi(optarg);
......@@ -219,6 +227,9 @@ main(int argc, char* argv[]) {
case 'o':
origin = optarg;
break;
case 'd':
debug_log = true;
break;
case '?':
default:
usage();
......@@ -232,9 +243,9 @@ main(int argc, char* argv[]) {
const char* const datasrc_file = argv[0];
const char* const query_data_file = argv[1];
// We disable logging to avoid unwanted noise. (We may eventually want to
// make it more configurable)
initLogger("query-bench", isc::log::NONE);
// By default disable logging to avoid unwanted noise.
initLogger("query-bench", debug_log ? isc::log::DEBUG : isc::log::NONE,
isc::log::MAX_DEBUG_LEVEL, NULL);
DataSrcType datasrc_type = SQLITE3;
if (strcmp(opt_datasrc_type, "sqlite3") == 0) {
......@@ -251,34 +262,39 @@ main(int argc, char* argv[]) {
return (1);
}
BenchQueries queries;
loadQueryData(query_data_file, queries, RRClass::IN());
OutputBuffer buffer(4096);
Message message(Message::PARSE);
try {
BenchQueries queries;
loadQueryData(query_data_file, queries, RRClass::IN());
OutputBuffer buffer(4096);
Message message(Message::PARSE);
cout << "Parameters:" << endl;
cout << " Iterations: " << iteration << endl;
cout << " Data Source: type=" << opt_datasrc_type << ", file=" <<
datasrc_file << endl;
if (origin != NULL) {
cout << " Origin: " << origin << endl;
}
cout << " Query data: file=" << query_data_file << " (" << queries.size()
<< " queries)" << endl << endl;
cout << "Parameters:" << endl;
cout << " Iterations: " << iteration << endl;
cout << " Data Source: type=" << opt_datasrc_type << ", file=" <<
datasrc_file << endl;
if (origin != NULL) {
cout << " Origin: " << origin << endl;
}
cout << " Query data: file=" << query_data_file << " ("
<< queries.size() << " queries)" << endl << endl;
switch (datasrc_type) {
case SQLITE3:
cout << "Benchmark with SQLite3" << endl;
BenchMark<Sqlite3QueryBenchMark>(
iteration, Sqlite3QueryBenchMark(datasrc_file, queries,
message, buffer));
break;
case MEMORY:
cout << "Benchmark with In Memory Data Source" << endl;
BenchMark<MemoryQueryBenchMark>(
iteration, MemoryQueryBenchMark(datasrc_file, origin, queries,
message, buffer));
break;
switch (datasrc_type) {
case SQLITE3:
cout << "Benchmark with SQLite3" << endl;
BenchMark<Sqlite3QueryBenchMark>(
iteration, Sqlite3QueryBenchMark(datasrc_file, queries,
message, buffer));
break;
case MEMORY:
cout << "Benchmark with In Memory Data Source" << endl;
BenchMark<MemoryQueryBenchMark>(
iteration, MemoryQueryBenchMark(datasrc_file, origin, queries,
message, buffer));
break;
}
} catch (const std::exception& ex) {
cout << "Test unexpectedly failed: " << ex.what() << endl;
return (1);
}
return (0);
......
......@@ -98,16 +98,27 @@ public:
/// \c AuthCommandError when it encounters an internal error, such as
/// semantics error on the command arguments.
///
/// This method should return the execution result in the form of
/// \c ConstElementPtr. It will be transparently used as the return
/// value from the command handler called from the corresponding
/// \c CCSession object. For a successful completion of the command,
/// it should suffice to return the return value of
/// \c isc::config::createAnswer() with no argument.
///