Commit 6fe86386 authored by Mukund Sivaraman's avatar Mukund Sivaraman
Browse files

Merge branch 'master' into trac2503

Conflicts:
	src/lib/datasrc/tests/memory/testdata/Makefile.am
parents 3de3abb8 cbb1e342
522. [func]* jelte
Configuration of TSIG keys for b10-xfrin has changed; instead of
specifying the full TSIG key (<name>:<base64>:<algo>) it now expects
just the name, and uses the global TSIG Key Ring like all the other
components (configuration list /tsig_keys/keys).
Note: this is not automatically updated, so if you use TSIG in
xfrin, you need to update your configuration.
(Trac #1351, git e65b7b36f60f14b7abe083da411e6934cdfbae7a)
521. [func] marcin
Implemented definitions for DHCPv6 standard options identified
by codes up to 48. These definitions are now used by the DHCPv6
server to create instances of options being sent to a client.
(Trac #2491, git 0a4faa07777189ed9c25211987a1a9b574015a95)
520. [func] jelte
The system no longer prints initial log messages to stdout
regardless of what logging configuration is present, but it
temporarily stores any log messages until the configuration is
processed. If there is no specific configuration, or if the
configuration cannot be accessed, it will still fall back to stdout.
Note that there are still a few instances where output is printed,
these shall be addressed separately.
Note also that, currently, in case it falls back to stdout (such as
when it cannot connect to b10-cfgmgr), all log messages are always
printed (including debug messages), regardless of whether -v was
used. This shall also be addressed in a future change.
(Trac #2445, git 74a0abe5a6d10b28e4a3e360e87b129c232dea68)
519. [bug] muks
Fixed a problem in inmem NSEC lookup which caused returning an
incorrect NSEC record or (in rare cases) assert failures
when a non-existent domain was queried, which was a sub-domain of
a domain that existed.
(Trac #2504, git 835553eb309d100b062051f7ef18422d2e8e3ae4)
518. [func] stephen
Extend DHCP MySQL backend to handle IPv4 addresses.
(Trac #2404, git ce7db48d3ff5d5aad12b1da5e67ae60073cb2607)
517. [func] stephen
Added IOAddress::toBytes() to get byte representation of address.
Also added convenience methods for V4/V6 address determination.
(Trac #2396, git c23f87e8ac3ea781b38d688f8f7b58539f85e35a)
516. [bug] marcin
Fixed 'make distcheck' failure when running perfdhcp unit tests.
The unit tests used to read files from the folder specified
with the path relative to current folder, thus when the test was
run from a different folder the files could not be found.
(Trac #2479, git 4e8325e1b309f1d388a3055ec1e1df98c377f383)
515. [bug] jinmei
The in-memory data source now accepts an RRSIG provided without
a covered RRset in loading. A subsequent query for its owner name
......
......@@ -903,11 +903,12 @@ AC_SUBST(MULTITHREADING_FLAG)
#
GTEST_LDFLAGS=
GTEST_LDADD=
# TODO: set DISTCHECK_GTEST_CONFIGURE_FLAG for --with-gtest too
DISTCHECK_GTEST_CONFIGURE_FLAG=
if test "x$enable_gtest" = "xyes" ; then
DISTCHECK_GTEST_CONFIGURE_FLAG="--with-gtest=$gtest_path"
if test -n "$with_gtest_source" ; then
if test "x$GTEST_SOURCE" = "xyes" ; then
......@@ -1375,6 +1376,7 @@ AC_OUTPUT([doc/version.ent
src/lib/log/tests/console_test.sh
src/lib/log/tests/destination_test.sh
src/lib/log/tests/init_logger_test.sh
src/lib/log/tests/buffer_logger_test.sh
src/lib/log/tests/local_file_test.sh
src/lib/log/tests/logger_lock_test.sh
src/lib/log/tests/severity_test.sh
......
......@@ -580,8 +580,8 @@ INPUT = ../src/lib/exceptions ../src/lib/cc \
../src/lib/testutils ../src/lib/cache ../src/lib/server_common/ \
../src/bin/sockcreator/ ../src/lib/util/ ../src/lib/util/io/ \
../src/lib/util/threads/ ../src/lib/resolve ../src/lib/acl \
../src/lib/statistics ../src/bin/dhcp6 ../src/lib/dhcp ../src/bin/dhcp4 \
../tests/tools/perfdhcp devel
../src/lib/statistics ../src/bin/dhcp6 ../src/lib/dhcp ../src/lib/dhcpsrv \
../src/bin/dhcp4 ../tests/tools/perfdhcp devel
# This tag can be used to specify the character encoding of the source files
# that doxygen parses. Internally doxygen uses the UTF-8 encoding, which is
......
......@@ -472,7 +472,7 @@ var/
<title>Packages</title>
<para>
Some operating systems or softare package vendors may
Some operating systems or software package vendors may
provide ready-to-use, pre-built software packages for
the BIND 10 suite.
Installing a pre-built package means you do not need to
......@@ -758,7 +758,7 @@ as a dependency earlier -->
If the configure fails, it may be due to missing or old
dependencies.
</para>
<note>
<para>For notes on configuring and building DHCPv6 with MySQL see <xref linkend="dhcp6-install">.</xref></para>
</note>
......@@ -1841,10 +1841,8 @@ config set /Boss/components/b10-zonemgr/kind dispensable
<para>
The key ring lives in the configuration in "tsig_keys/keys". Most of
the system uses the keys from there &mdash; ACLs, authoritative server to
sign responses to signed queries, and <command>b10-xfrout</command>
to sign transfers. The <command>b10-xfrin</command> uses its own
configuration for keys, but that will be fixed in Trac ticket
<ulink url="http://bind10.isc.org/ticket/1351">#1351</ulink>.
sign responses to signed queries, and <command>b10-xfrin</command>
and <command>b10-xfrout</command> to sign transfers.
</para>
<para>
......@@ -2157,7 +2155,7 @@ AND_MATCH := "ALL": [ RULE_RAW, RULE_RAW, ... ]
you indicate that the system is not usable without the
component and if such component fails, the system shuts
down no matter when the failure happened. This is the
behaviour of the core components (the ones you can't turn
behavior of the core components (the ones you can't turn
off), but you can declare any other components as core as
well if you wish (but you can turn these off, they just
can't fail).
......@@ -2721,6 +2719,15 @@ TODO
</para>
</section>
<section>
<title>TSIG</title>
If you want to use TSIG for incoming transfers, a system wide TSIG
key ring must be configured (see <xref linkend="tsig-key-ring"/>).
To specify a key to use, set tsig_key value to the name of the key
to use from the key ring.
&gt; <userinput>config set Xfrin/zones[0]/tsig_key "<option>example.key</option>"</userinput>
</section>
<section>
<title>Enabling IXFR</title>
<para>
......
......@@ -147,7 +147,7 @@ main(int argc, char* argv[]) {
// Initialize logging. If verbose, we'll use maximum verbosity.
isc::log::initLogger(AUTH_NAME,
(verbose ? isc::log::DEBUG : isc::log::INFO),
isc::log::MAX_DEBUG_LEVEL, NULL);
isc::log::MAX_DEBUG_LEVEL, NULL, true);
int ret = 0;
......@@ -256,7 +256,9 @@ main(int argc, char* argv[]) {
// If we haven't registered callback for data sources, this will be just
// no-op.
config_session->removeRemoteConfig("data_sources");
if (config_session != NULL) {
config_session->removeRemoteConfig("data_sources");
}
delete xfrin_session;
delete config_session;
......
......@@ -225,7 +225,7 @@ createBuiltinVersionResponse(const qid_t qid, vector<uint8_t>& data) {
message.setHeaderFlag(Message::HEADERFLAG_AA);
RRsetPtr rrset_version = RRsetPtr(new RRset(version_name, RRClass::CH(),
RRType::TXT(), RRTTL(0)));
rrset_version->addRdata(generic::TXT(PACKAGE_STRING));
rrset_version->addRdata(generic::TXT("\"" PACKAGE_STRING "\""));
message.addRRset(Message::SECTION_ANSWER, rrset_version);
RRsetPtr rrset_version_ns = RRsetPtr(new RRset(apex_name, RRClass::CH(),
......
......@@ -48,7 +48,7 @@ else:
PREFIX = "@prefix@"
DATAROOTDIR = "@datarootdir@"
SPECFILE_LOCATION = "@datadir@/@PACKAGE@/bob.spec".replace("${datarootdir}", DATAROOTDIR).replace("${prefix}", PREFIX)
import subprocess
import signal
import re
......@@ -76,7 +76,7 @@ import isc.bind10.socket_cache
import libutil_io_python
import tempfile
isc.log.init("b10-boss")
isc.log.init("b10-boss", buffer=True)
logger = isc.log.Logger("boss")
# Pending system-wide debug level definitions, the ones we
......@@ -166,14 +166,14 @@ class ProcessStartError(Exception): pass
class BoB:
"""Boss of BIND class."""
def __init__(self, msgq_socket_file=None, data_path=None,
config_filename=None, clear_config=False,
verbose=False, nokill=False, setuid=None, setgid=None,
username=None, cmdctl_port=None, wait_time=10):
"""
Initialize the Boss of BIND. This is a singleton (only one can run).
The msgq_socket_file specifies the UNIX domain socket file that the
msgq process listens on. If verbose is True, then the boss reports
what it is doing.
......@@ -216,6 +216,12 @@ class BoB:
self.clear_config = clear_config
self.cmdctl_port = cmdctl_port
self.wait_time = wait_time
self.msgq_timeout = 5
# _run_under_unittests is only meant to be used when testing. It
# bypasses execution of some code to help with testing.
self._run_under_unittests = False
self._component_configurator = isc.bind10.component.Configurator(self,
isc.bind10.special_component.get_specials())
# The priorities here make them start in the correct order. First
......@@ -332,6 +338,7 @@ class BoB:
"""
logger.info(BIND10_KILLING_ALL_PROCESSES)
self.__kill_children(True)
self.components = {}
def _read_bind10_config(self):
"""
......@@ -400,7 +407,7 @@ class BoB:
logger.error(BIND10_STARTUP_UNEXPECTED_MESSAGE, msg)
except:
logger.error(BIND10_STARTUP_UNRECOGNISED_MESSAGE, msg)
return False
# The next few methods start the individual processes of BIND-10. They
......@@ -408,21 +415,34 @@ class BoB:
# raised which is caught by the caller of start_all_processes(); this kills
# processes started up to that point before terminating the program.
def _make_process_info(self, name, args, env,
dev_null_stdout=False, dev_null_stderr=False):
"""
Wrapper around ProcessInfo(), useful to override
ProcessInfo() creation during testing.
"""
return ProcessInfo(name, args, env, dev_null_stdout, dev_null_stderr)
def start_msgq(self):
"""
Start the message queue and connect to the command channel.
"""
self.log_starting("b10-msgq")
msgq_proc = ProcessInfo("b10-msgq", ["b10-msgq"], self.c_channel_env,
True, not self.verbose)
msgq_proc = self._make_process_info("b10-msgq", ["b10-msgq"],
self.c_channel_env,
True, not self.verbose)
msgq_proc.spawn()
self.log_started(msgq_proc.pid)
# Now connect to the c-channel
cc_connect_start = time.time()
while self.cc_session is None:
# if we are run under unittests, break
if self._run_under_unittests:
break
# if we have been trying for "a while" give up
if (time.time() - cc_connect_start) > 5:
if (time.time() - cc_connect_start) > self.msgq_timeout:
logger.error(BIND10_CONNECTING_TO_CC_FAIL)
raise CChannelConnectError("Unable to connect to c-channel after 5 seconds")
......@@ -434,7 +454,8 @@ class BoB:
# Subscribe to the message queue. The only messages we expect to receive
# on this channel are once relating to process startup.
self.cc_session.group_subscribe("Boss")
if self.cc_session is not None:
self.cc_session.group_subscribe("Boss")
return msgq_proc
......@@ -450,13 +471,14 @@ class BoB:
args.append("--config-filename=" + self.config_filename)
if self.clear_config:
args.append("--clear-config")
bind_cfgd = ProcessInfo("b10-cfgmgr", args,
self.c_channel_env)
bind_cfgd = self._make_process_info("b10-cfgmgr", args,
self.c_channel_env)
bind_cfgd.spawn()
self.log_started(bind_cfgd.pid)
# Wait for the configuration manager to start up as subsequent initialization
# cannot proceed without it. The time to wait can be set on the command line.
# Wait for the configuration manager to start up as
# subsequent initialization cannot proceed without it. The
# time to wait can be set on the command line.
time_remaining = self.wait_time
msg, env = self.cc_session.group_recvmsg()
while time_remaining > 0 and not self.process_running(msg, "ConfigManager"):
......@@ -464,7 +486,7 @@ class BoB:
time.sleep(1)
time_remaining = time_remaining - 1
msg, env = self.cc_session.group_recvmsg()
if not self.process_running(msg, "ConfigManager"):
raise ProcessStartError("Configuration manager process has not started")
......@@ -481,7 +503,7 @@ class BoB:
process, the log_starting/log_started methods are not used.
"""
logger.info(BIND10_STARTING_CC)
self.ccs = isc.config.ModuleCCSession(SPECFILE_LOCATION,
self.ccs = isc.config.ModuleCCSession(SPECFILE_LOCATION,
self.config_handler,
self.command_handler,
socket_file = self.msgq_socket_file)
......@@ -499,7 +521,7 @@ class BoB:
The port and address arguments are for log messages only.
"""
self.log_starting(name, port, address)
newproc = ProcessInfo(name, args, c_channel_env)
newproc = self._make_process_info(name, args, c_channel_env)
newproc.spawn()
self.log_started(newproc.pid)
return newproc
......@@ -611,7 +633,6 @@ class BoB:
if self.msgq_socket_file is not None:
c_channel_env["BIND10_MSGQ_SOCKET_FILE"] = self.msgq_socket_file
logger.debug(DBG_PROCESS, BIND10_CHECK_MSGQ_ALREADY_RUNNING)
# try to connect, and if we can't wait a short while
try:
self.cc_session = isc.cc.Session(self.msgq_socket_file)
logger.fatal(BIND10_MSGQ_ALREADY_RUNNING)
......@@ -679,7 +700,7 @@ class BoB:
except:
pass
# XXX: some delay probably useful... how much is uncertain
# I have changed the delay from 0.5 to 1, but sometime it's
# I have changed the delay from 0.5 to 1, but sometime it's
# still not enough.
time.sleep(1)
self.reap_children()
......@@ -728,17 +749,19 @@ class BoB:
return os.waitpid(-1, os.WNOHANG)
def reap_children(self):
"""Check to see if any of our child processes have exited,
and note this for later handling.
"""Check to see if any of our child processes have exited,
and note this for later handling.
"""
while True:
try:
(pid, exit_status) = self._get_process_exit_status()
except OSError as o:
if o.errno == errno.ECHILD: break
if o.errno == errno.ECHILD:
break
# XXX: should be impossible to get any other error here
raise
if pid == 0: break
if pid == 0:
break
if pid in self.components:
# One of the components we know about. Get information on it.
component = self.components.pop(pid)
......@@ -760,11 +783,11 @@ class BoB:
"""
Restart any dead processes:
* Returns the time when the next process is ready to be restarted.
* Returns the time when the next process is ready to be restarted.
* If the server is shutting down, returns 0.
* If there are no processes, returns None.
The values returned can be safely passed into select() as the
The values returned can be safely passed into select() as the
timeout value.
"""
......@@ -909,8 +932,10 @@ class BoB:
"""
if self._srv_socket is not None:
self._srv_socket.close()
os.remove(self._socket_path)
os.rmdir(self._tmpdir)
if os.path.exists(self._socket_path):
os.remove(self._socket_path)
if os.path.isdir(self._tmpdir):
os.rmdir(self._tmpdir)
def _srv_accept(self):
"""
......@@ -1006,7 +1031,7 @@ boss_of_bind = None
def reaper(signal_number, stack_frame):
"""A child process has died (SIGCHLD received)."""
# don't do anything...
# don't do anything...
# the Python signal handler has been set up to write
# down a pipe, waking up our select() bit
pass
......@@ -1173,7 +1198,7 @@ and the created lock file must be writable for that user.
except KeyError:
pass
# Next try getting information about the user, assuming user name
# Next try getting information about the user, assuming user name
# passed.
# If the information is both a valid user name and user number, we
# prefer the name because we try it second. A minor point, hopefully.
......
......@@ -25,6 +25,7 @@ import bind10_src
import unittest
import sys
import os
import os.path
import copy
import signal
import socket
......@@ -34,6 +35,7 @@ import isc
import isc.log
import isc.bind10.socket_cache
import errno
import random
from isc.testutils.parse_args import TestOptParser, OptsError
from isc.testutils.ccsession_mock import MockModuleCCSession
......@@ -366,6 +368,53 @@ class TestBoB(unittest.TestCase):
self.assertEqual(creator, bob._socket_cache._creator)
self.assertRaises(ValueError, bob.set_creator, creator)
def test_socket_srv(self):
"""Tests init_socket_srv() and remove_socket_srv() work as expected."""
bob = BoB()
self.assertIsNone(bob._srv_socket)
self.assertIsNone(bob._tmpdir)
self.assertIsNone(bob._socket_path)
bob.init_socket_srv()
self.assertIsNotNone(bob._srv_socket)
self.assertNotEqual(-1, bob._srv_socket.fileno())
self.assertEqual(os.path.join(bob._tmpdir, 'sockcreator'),
bob._srv_socket.getsockname())
self.assertIsNotNone(bob._tmpdir)
self.assertTrue(os.path.isdir(bob._tmpdir))
self.assertIsNotNone(bob._socket_path)
self.assertTrue(os.path.exists(bob._socket_path))
# Check that it's possible to connect to the socket file (this
# only works if the socket file exists and the server listens on
# it).
s = socket.socket(socket.AF_UNIX)
try:
s.connect(bob._socket_path)
can_connect = True
s.close()
except socket.error as e:
can_connect = False
self.assertTrue(can_connect)
bob.remove_socket_srv()
self.assertEqual(-1, bob._srv_socket.fileno())
self.assertFalse(os.path.exists(bob._socket_path))
self.assertFalse(os.path.isdir(bob._tmpdir))
# These should not fail either:
# second call
bob.remove_socket_srv()
bob._srv_socket = None
bob.remove_socket_srv()
def test_init_alternate_socket(self):
bob = BoB("alt_socket_file")
self.assertEqual(bob.verbose, False)
......@@ -461,6 +510,22 @@ class TestBoB(unittest.TestCase):
self.assertEqual({'command': ['shutdown', {'pid': 42}]},
bob.cc_session.msg)
# Mock class for testing BoB's usage of ProcessInfo
class MockProcessInfo:
def __init__(self, name, args, env={}, dev_null_stdout=False,
dev_null_stderr=False):
self.name = name
self.args = args
self.env = env
self.dev_null_stdout = dev_null_stdout
self.dev_null_stderr = dev_null_stderr
self.process = None
self.pid = None
def spawn(self):
# set some pid (only used for testing that it is not None anymore)
self.pid = 42147
# Class for testing the BoB without actually starting processes.
# This is used for testing the start/stop components routines and
# the BoB commands.
......@@ -490,6 +555,7 @@ class MockBob(BoB):
self.c_channel_env = {}
self.components = { }
self.creator = False
self.get_process_exit_status_called = False
class MockSockCreator(isc.bind10.component.Component):
def __init__(self, process, boss, kind, address=None, params=None):
......@@ -661,6 +727,52 @@ class MockBob(BoB):
del self.components[12]
self.cmdctl = False
def _get_process_exit_status(self):
if self.get_process_exit_status_called:
return (0, 0)
self.get_process_exit_status_called = True
return (53, 0)
def _get_process_exit_status_unknown_pid(self):
if self.get_process_exit_status_called:
return (0, 0)
self.get_process_exit_status_called = True
return (42, 0)
def _get_process_exit_status_raises_oserror_echild(self):
raise OSError(errno.ECHILD, 'Mock error')
def _get_process_exit_status_raises_oserror_other(self):
raise OSError(0, 'Mock error')
def _get_process_exit_status_raises_other(self):
raise Exception('Mock error')
def _make_mock_process_info(self, name, args, c_channel_env,
dev_null_stdout=False, dev_null_stderr=False):
return MockProcessInfo(name, args, c_channel_env,
dev_null_stdout, dev_null_stderr)
class MockBobSimple(BoB):
def __init__(self):
BoB.__init__(self)
# Set which process has been started
self.started_process_name = None
self.started_process_args = None
self.started_process_env = None
def _make_mock_process_info(self, name, args, c_channel_env,
dev_null_stdout=False, dev_null_stderr=False):
return MockProcessInfo(name, args, c_channel_env,
dev_null_stdout, dev_null_stderr)
def start_process(self, name, args, c_channel_env, port=None,
address=None):
self.started_process_name = name
self.started_process_args = args
self.started_process_env = c_channel_env
return None
class TestStartStopProcessesBob(unittest.TestCase):
"""
Check that the start_all_components method starts the right combination
......@@ -930,6 +1042,9 @@ class MockComponent:
self.pid = lambda: pid
self.address = lambda: address
self.restarted = False
self.forceful = False
self.running = True
self.has_failed = False
def get_restart_time(self):
return 0 # arbitrary dummy value
......@@ -938,6 +1053,15 @@ class MockComponent:
self.restarted = True
return True
def is_running(self):
return self.running
def failed(self, status):
return self.has_failed
def kill(self, forceful):
self.forceful = forceful
class TestBossCmd(unittest.TestCase):
def test_ping(self):
"""
......@@ -1107,6 +1231,20 @@ class TestBossComponents(unittest.TestCase):
'process': 'cat'
}
}
self._tmp_time = None
self._tmp_sleep = None
self._tmp_module_cc_session = None
self._tmp_cc_session = None
def tearDown(self):
if self._tmp_time is not None:
time.time = self._tmp_time
if self._tmp_sleep is not None:
time.sleep = self._tmp_sleep
if self._tmp_module_cc_session is not None:
isc.config.ModuleCCSession = self._tmp_module_cc_session
if self._tmp_cc_session is not None:
isc.cc.Session = self._tmp_cc_session
def __unary_hook(self, param):
"""
......@@ -1324,14 +1462,618 @@ class TestBossComponents(unittest.TestCase):
bob._component_configurator._components['test'] = (None, component)
self.__setup_restart(bob, component)
self.assertTrue(component.restarted)
self.assertFalse(component in bob.components_to_restart)
self.assertNotIn(component, bob.components_to_restart)
# Remove the component from the configuration. It won't be restarted
# even if scheduled, nor will remain in the to-be-restarted list.
del bob._component_configurator._components['test']
self.__setup_restart(bob, component)
self.assertFalse(component.restarted)
self.assertFalse(component in bob.components_to_restart)
self.assertNotIn(component, bob.components_to_restart)
def test_get_processes(self):
'''Test that procsses are returned correctly, sorted by pid.'''
bob = MockBob()
pids = list(range(0, 20))
random.shuffle(pids)
for i in range(0, 20):
pid = pids[i]
component = MockComponent('test' + str(pid), pid,
'Test' + str(pid))
bob.components[pid] = component
process_list = bob.get_processes()
self.assertEqual(20, len(process_list))
last_pid = -1
for process in process_list:
pid = process[0]
self.assertLessEqual(last_pid, pid)
last_pid = pid
self.assertEqual([pid, 'test' + str(pid), 'Test' + str(pid)],
process)
def _test_reap_children_helper(self, runna