Commit 58996536 authored by Jelte Jansen's avatar Jelte Jansen
Browse files
parents cb86d164 f279d996
......@@ -904,9 +904,10 @@ AC_OUTPUT([doc/version.ent
src/bin/zonemgr/run_b10-zonemgr.sh
src/bin/stats/stats.py
src/bin/stats/stats_httpd.py
src/bin/bind10/bind10.py
src/bin/bind10/bind10_src.py
src/bin/bind10/run_bind10.sh
src/bin/bind10/tests/bind10_test.py
src/bin/bind10/tests/sockcreator_test.py
src/bin/bindctl/run_bindctl.sh
src/bin/bindctl/bindctl_main.py
src/bin/bindctl/tests/bindctl_test
......
SUBDIRS = . tests
sbin_SCRIPTS = bind10
CLEANFILES = bind10 bind10.pyc bind10_messages.py bind10_messages.pyc
CLEANFILES = bind10 bind10_src.pyc bind10_messages.py bind10_messages.pyc \
sockcreator.pyc
python_PYTHON = __init__.py sockcreator.py
pythondir = $(pyexecdir)/bind10
pkglibexecdir = $(libexecdir)/@PACKAGE@
pyexec_DATA = bind10_messages.py
......@@ -24,9 +28,9 @@ bind10_messages.py: bind10_messages.mes
$(top_builddir)/src/lib/log/compiler/message -p $(top_srcdir)/src/bin/bind10/bind10_messages.mes
# this is done here since configure.ac AC_OUTPUT doesn't expand exec_prefix
bind10: bind10.py
bind10: bind10_src.py
$(SED) -e "s|@@PYTHONPATH@@|@pyexecdir@|" \
-e "s|@@LIBEXECDIR@@|$(pkglibexecdir)|" bind10.py >$@
-e "s|@@LIBEXECDIR@@|$(pkglibexecdir)|" bind10_src.py >$@
chmod a+x $@
pytest:
......
......@@ -32,15 +32,15 @@ started according to the configuration.
The boss process was started with the -u option, to drop root privileges
and continue running as the specified user, but the user is unknown.
% BIND10_KILLING_ALL_PROCESSES killing all started processes
The boss module was not able to start every process it needed to start
during startup, and will now kill the processes that did get started.
% BIND10_KILL_PROCESS killing process %1
The boss module is sending a kill signal to process with the given name,
as part of the process of killing all started processes during a failed
startup, as described for BIND10_KILLING_ALL_PROCESSES
% BIND10_KILLING_ALL_PROCESSES killing all started processes
The boss module was not able to start every process it needed to start
during startup, and will now kill the processes that did get started.
% BIND10_MSGQ_ALREADY_RUNNING msgq daemon already running, cannot start
There already appears to be a message bus daemon running. Either an
old process was not shut down correctly, and needs to be killed, or
......@@ -113,12 +113,49 @@ it shall send SIGKILL signals to the processes still alive.
All child processes have been stopped, and the boss process will now
stop itself.
% BIND10_START_AS_NON_ROOT starting %1 as a user, not root. This might fail.
The given module is being started or restarted without root privileges.
If the module needs these privileges, it may have problems starting.
Note that this issue should be resolved by the pending 'socket-creator'
process; once that has been implemented, modules should not need root
privileges anymore. See tickets #800 and #801 for more information.
% BIND10_SOCKCREATOR_BAD_CAUSE unknown error cause from socket creator: %1
The socket creator reported an error when creating a socket. But the function
which failed is unknown (not one of 'S' for socket or 'B' for bind).
% BIND10_SOCKCREATOR_BAD_RESPONSE unknown response for socket request: %1
The boss requested a socket from the creator, but the answer is unknown. This
looks like a programmer error.
% BIND10_SOCKCREATOR_CRASHED the socket creator crashed
The socket creator terminated unexpectadly. It is not possible to restart it
(because the boss already gave up root privileges), so the system is going
to terminate.
% BIND10_SOCKCREATOR_EOF eof while expecting data from socket creator
There should be more data from the socket creator, but it closed the socket.
It probably crashed.
% BIND10_SOCKCREATOR_INIT initializing socket creator parser
The boss module initializes routines for parsing the socket creator
protocol.
% BIND10_SOCKCREATOR_KILL killing the socket creator
The socket creator is being terminated the aggressive way, by sending it
sigkill. This should not happen usually.
% BIND10_SOCKCREATOR_TERMINATE terminating socket creator
The boss module sends a request to terminate to the socket creator.
% BIND10_SOCKCREATOR_TRANSPORT_ERROR transport error when talking to the socket creator: %1
Either sending or receiving data from the socket creator failed with the given
error. The creator probably crashed or some serious OS-level problem happened,
as the communication happens only on local host.
% BIND10_SOCKET_CREATED successfully created socket %1
The socket creator successfully created and sent a requested socket, it has
the given file number.
% BIND10_SOCKET_ERROR error on %1 call in the creator: %2/%3
The socket creator failed to create the requested socket. It failed on the
indicated OS API function with given error.
% BIND10_SOCKET_GET requesting socket [%1]:%2 of type %3 from the creator
The boss forwards a request for a socket to the socket creator.
% BIND10_STARTED_PROCESS started %1
The given process has successfully been started.
......@@ -147,6 +184,13 @@ All modules have been successfully started, and BIND 10 is now running.
There was a fatal error when BIND10 was trying to start. The error is
shown, and BIND10 will now shut down.
% BIND10_START_AS_NON_ROOT starting %1 as a user, not root. This might fail.
The given module is being started or restarted without root privileges.
If the module needs these privileges, it may have problems starting.
Note that this issue should be resolved by the pending 'socket-creator'
process; once that has been implemented, modules should not need root
privileges anymore. See tickets #800 and #801 for more information.
% BIND10_STOP_PROCESS asking %1 to shut down
The boss module is sending a shutdown command to the given module over
the message channel.
......@@ -154,4 +198,3 @@ the message channel.
% BIND10_UNKNOWN_CHILD_PROCESS_ENDED unknown child pid %1 exited
An unknown child process has exited. The PID is printed, but no further
action will be taken by the boss process.
......@@ -67,6 +67,7 @@ import isc.util.process
import isc.net.parse
import isc.log
from bind10_messages import *
import bind10.sockcreator
isc.log.init("b10-boss")
logger = isc.log.Logger("boss")
......@@ -248,6 +249,7 @@ class BoB:
self.config_filename = config_filename
self.cmdctl_port = cmdctl_port
self.brittle = brittle
self.sockcreator = None
def config_handler(self, new_config):
# If this is initial update, don't do anything now, leave it to startup
......@@ -333,6 +335,20 @@ class BoB:
"Unknown command")
return answer
def start_creator(self):
self.curproc = 'b10-sockcreator'
self.sockcreator = bind10.sockcreator.Creator("@@LIBEXECDIR@@:" +
os.environ['PATH'])
def stop_creator(self, kill=False):
if self.sockcreator is None:
return
if kill:
self.sockcreator.kill()
else:
self.sockcreator.terminate()
self.sockcreator = None
def kill_started_processes(self):
"""
Called as part of the exception handling when a process fails to
......@@ -341,6 +357,8 @@ class BoB:
"""
logger.info(BIND10_KILLING_ALL_PROCESSES)
self.stop_creator(True)
for pid in self.processes:
logger.info(BIND10_KILL_PROCESS, self.processes[pid].name)
self.processes[pid].process.kill()
......@@ -571,6 +589,11 @@ class BoB:
Starts up all the processes. Any exception generated during the
starting of the processes is handled by the caller.
"""
# The socket creator first, as it is the only thing that needs root
self.start_creator()
# TODO: Once everything uses the socket creator, we can drop root
# privileges right now
c_channel_env = self.c_channel_env
self.start_msgq(c_channel_env)
self.start_cfgmgr(c_channel_env)
......@@ -660,6 +683,8 @@ class BoB:
self.cc_session.group_sendmsg(cmd, "Zonemgr", "Zonemgr")
self.cc_session.group_sendmsg(cmd, "Stats", "Stats")
self.cc_session.group_sendmsg(cmd, "StatsHttpd", "StatsHttpd")
# Terminate the creator last
self.stop_creator()
def stop_process(self, process, recipient):
"""
......@@ -746,7 +771,14 @@ class BoB:
# XXX: should be impossible to get any other error here
raise
if pid == 0: break
if pid in self.processes:
if self.sockcreator is not None and self.sockcreator.pid() == pid:
# This is the socket creator, started and terminated
# differently. This can't be restarted.
if self.runnable:
logger.fatal(BIND10_SOCKCREATOR_CRASHED)
self.sockcreator = None
self.runnable = False
elif pid in self.processes:
# One of the processes we know about. Get information on it.
proc_info = self.processes.pop(pid)
proc_info.restart_schedule.set_run_stop_time()
......
# Copyright (C) 2011 Internet Systems Consortium, Inc. ("ISC")
#
# Permission to use, copy, modify, and distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
import socket
import struct
import os
import subprocess
from bind10_messages import *
from libutil_io_python import recv_fd
logger = isc.log.Logger("boss")
"""
Module that comunicates with the privileged socket creator (b10-sockcreator).
"""
class CreatorError(Exception):
"""
Exception for socket creator related errors.
It has two members: fatal and errno and they are just holding the values
passed to the __init__ function.
"""
def __init__(self, message, fatal, errno=None):
"""
Creates the exception. The message argument is the usual string.
The fatal one tells if the error is fatal (eg. the creator crashed)
and errno is the errno value returned from socket creator, if
applicable.
"""
Exception.__init__(self, message)
self.fatal = fatal
self.errno = errno
class Parser:
"""
This class knows the sockcreator language. It creates commands, sends them
and receives the answers and parses them.
It does not start it, the communication channel must be provided.
In theory, anything here can throw a fatal CreatorError exception, but it
happens only in case something like the creator process crashes. Any other
occasions are mentioned explicitly.
"""
def __init__(self, creator_socket):
"""
Creates the parser. The creator_socket is socket to the socket creator
process that will be used for communication. However, the object must
have a read_fd() method to read the file descriptor. This slightly
unusual trick with modifying an object is used to easy up testing.
You can use WrappedSocket in production code to add the method to any
ordinary socket.
"""
self.__socket = creator_socket
logger.info(BIND10_SOCKCREATOR_INIT)
def terminate(self):
"""
Asks the creator process to terminate and waits for it to close the
socket. Does not return anything. Raises a CreatorError if there is
still data on the socket, if there is an error closing the socket,
or if the socket had already been closed.
"""
if self.__socket is None:
raise CreatorError('Terminated already', True)
logger.info(BIND10_SOCKCREATOR_TERMINATE)
try:
self.__socket.sendall(b'T')
# Wait for an EOF - it will return empty data
eof = self.__socket.recv(1)
if len(eof) != 0:
raise CreatorError('Protocol error - data after terminated',
True)
self.__socket = None
except socket.error as se:
self.__socket = None
raise CreatorError(str(se), True)
def get_socket(self, address, port, socktype):
"""
Asks the socket creator process to create a socket. Pass an address
(the isc.net.IPaddr object), port number and socket type (either
string "UDP", "TCP" or constant socket.SOCK_DGRAM or
socket.SOCK_STREAM.
Blocks until it is provided by the socket creator process (which
should be fast, as it is on localhost) and returns the file descriptor
number. It raises a CreatorError exception if the creation fails.
"""
if self.__socket is None:
raise CreatorError('Socket requested on terminated creator', True)
# First, assemble the request from parts
logger.info(BIND10_SOCKET_GET, address, port, socktype)
data = b'S'
if socktype == 'UDP' or socktype == socket.SOCK_DGRAM:
data += b'U'
elif socktype == 'TCP' or socktype == socket.SOCK_STREAM:
data += b'T'
else:
raise ValueError('Unknown socket type: ' + str(socktype))
if address.family == socket.AF_INET:
data += b'4'
elif address.family == socket.AF_INET6:
data += b'6'
else:
raise ValueError('Unknown address family in address')
data += struct.pack('!H', port)
data += address.addr
try:
# Send the request
self.__socket.sendall(data)
answer = self.__socket.recv(1)
if answer == b'S':
# Success!
result = self.__socket.read_fd()
logger.info(BIND10_SOCKET_CREATED, result)
return result
elif answer == b'E':
# There was an error, read the error as well
error = self.__socket.recv(1)
errno = struct.unpack('i',
self.__read_all(len(struct.pack('i',
0))))
if error == b'S':
cause = 'socket'
elif error == b'B':
cause = 'bind'
else:
self.__socket = None
logger.fatal(BIND10_SOCKCREATOR_BAD_CAUSE, error)
raise CreatorError('Unknown error cause' + str(answer), True)
logger.error(BIND10_SOCKET_ERROR, cause, errno[0],
os.strerror(errno[0]))
raise CreatorError('Error creating socket on ' + cause, False,
errno[0])
else:
self.__socket = None
logger.fatal(BIND10_SOCKCREATOR_BAD_RESPONSE, answer)
raise CreatorError('Unknown response ' + str(answer), True)
except socket.error as se:
self.__socket = None
logger.fatal(BIND10_SOCKCREATOR_TRANSPORT_ERROR, str(se))
raise CreatorError(str(se), True)
def __read_all(self, length):
"""
Keeps reading until length data is read or EOF or error happens.
EOF is considered error as well and throws a CreatorError.
"""
result = b''
while len(result) < length:
data = self.__socket.recv(length - len(result))
if len(data) == 0:
self.__socket = None
logger.fatal(BIND10_SOCKCREATOR_EOF)
raise CreatorError('Unexpected EOF', True)
result += data
return result
class WrappedSocket:
"""
This class wraps a socket and adds a read_fd method, so it can be used
for the Parser class conveniently. It simply copies all its guts into
itself and implements the method.
"""
def __init__(self, socket):
# Copy whatever can be copied from the socket
for name in dir(socket):
if name not in ['__class__', '__weakref__']:
setattr(self, name, getattr(socket, name))
# Keep the socket, so we can prevent it from being garbage-collected
# and closed before we are removed ourself
self.__orig_socket = socket
def read_fd(self):
"""
Read the file descriptor from the socket.
"""
return recv_fd(self.fileno())
# FIXME: Any idea how to test this? Starting an external process doesn't sound
# OK
class Creator(Parser):
"""
This starts the socket creator and allows asking for the sockets.
"""
def __init__(self, path):
(local, remote) = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM)
# Popen does not like, for some reason, having the same socket for
# stdin as well as stdout, so we dup it before passing it there.
remote2 = socket.fromfd(remote.fileno(), socket.AF_UNIX,
socket.SOCK_STREAM)
env = os.environ
env['PATH'] = path
self.__process = subprocess.Popen(['b10-sockcreator'], env=env,
stdin=remote.fileno(),
stdout=remote2.fileno())
remote.close()
remote2.close()
Parser.__init__(self, WrappedSocket(local))
def pid(self):
return self.__process.pid
def kill(self):
logger.warn(BIND10_SOCKCREATOR_KILL)
if self.__process is not None:
self.__process.kill()
self.__process = None
PYCOVERAGE_RUN = @PYCOVERAGE_RUN@
#PYTESTS = args_test.py bind10_test.py
# NOTE: this has a generated test found in the builddir
PYTESTS = bind10_test.py
EXTRA_DIST = $(PYTESTS)
PYTESTS = bind10_test.py sockcreator_test.py
# If necessary (rare cases), explicitly specify paths to dynamic libraries
# required by loadable python modules.
......@@ -21,7 +20,7 @@ endif
for pytest in $(PYTESTS) ; do \
echo Running test: $$pytest ; \
$(LIBRARY_PATH_PLACEHOLDER) \
env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_builddir)/src/bin/bind10 \
env PYTHONPATH=$(abs_top_srcdir)/src/lib/python:$(abs_top_builddir)/src/lib/python:$(abs_top_srcdir)/src/bin:$(abs_top_builddir)/src/bin/bind10:$(abs_top_builddir)/src/lib/util/io/.libs \
BIND10_MSGQ_SOCKET_FILE=$(abs_top_builddir)/msgq_socket \
$(PYCOVERAGE_RUN) $(abs_builddir)/$$pytest || exit ; \
done
......@@ -13,7 +13,7 @@
# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
from bind10 import ProcessInfo, BoB, parse_args, dump_pid, unlink_pid_file, _BASETIME
from bind10_src import ProcessInfo, BoB, parse_args, dump_pid, unlink_pid_file, _BASETIME
# XXX: environment tests are currently disabled, due to the preprocessor
# setup that we have now complicating the environment
......@@ -193,6 +193,13 @@ class MockBob(BoB):
self.cmdctl = False
self.c_channel_env = {}
self.processes = { }
self.creator = False
def start_creator(self):
self.creator = True
def stop_creator(self, kill=False):
self.creator = False
def read_bind10_config(self):
# Configuration options are set directly
......@@ -337,6 +344,7 @@ class TestStartStopProcessesBob(unittest.TestCase):
self.assertEqual(bob.msgq, core)
self.assertEqual(bob.cfgmgr, core)
self.assertEqual(bob.ccsession, core)
self.assertEqual(bob.creator, core)
self.assertEqual(bob.auth, auth)
self.assertEqual(bob.resolver, resolver)
self.assertEqual(bob.xfrout, auth)
......
# Copyright (C) 2011 Internet Systems Consortium, Inc. ("ISC")
#
# Permission to use, copy, modify, and distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
# This test file is generated .py.in -> .py just to be in the build dir,
# same as the rest of the tests. Saves a lot of stuff in makefile.
"""
Tests for the bind10.sockcreator module.
"""
import unittest
import struct
import socket
from isc.net.addr import IPAddr
import isc.log
from libutil_io_python import send_fd
from bind10.sockcreator import Parser, CreatorError, WrappedSocket
class FakeCreator:
"""
Class emulating the socket to the socket creator. It can be given expected
data to receive (and check) and responses to give to the Parser class
during testing.
"""
class InvalidPlan(Exception):
"""
Raised when someone wants to recv when sending is planned or vice
versa.
"""
pass
class InvalidData(Exception):
"""
Raises when the data passed to sendall are not the same as expected.
"""
pass
def __init__(self, plan):
"""
Create the object. The plan variable contains list of expected actions,
in form:
[('r', 'Data to return from recv'), ('s', 'Data expected on sendall'),
, ('d', 'File descriptor number to return from read_sock'), ('e',
None), ...]
It modifies the array as it goes.
"""
self.__plan = plan
def __get_plan(self, expected):
if len(self.__plan) == 0:
raise InvalidPlan('Nothing more planned')
(kind, data) = self.__plan[0]
if kind == 'e':
self.__plan.pop(0)
raise socket.error('False socket error')
if kind != expected:
raise InvalidPlan('Planned ' + kind + ', but ' + expected +
'requested')
return data
def recv(self, maxsize):
"""
Emulate recv. Returs maxsize bytes from the current recv plan. If
there are data left from previous recv call, it is used first.
If no recv is planned, raises InvalidPlan.
"""
data = self.__get_plan('r')
result, rest = data[:maxsize], data[maxsize:]
if len(rest) > 0:
self.__plan[0] = ('r', rest)
else:
self.__plan.pop(0)
return result
def read_fd(self):
"""
Emulate the reading of file descriptor. Returns one from a plan.
It raises InvalidPlan if no socket is planned now.
"""
fd = self.__get_plan('f')
self.__plan.pop(0)
return fd
def sendall(self, data):
"""
Checks that the data passed are correct according to plan. It raises
InvalidData if the data differs or InvalidPlan when sendall is not
expected.
"""
planned = self.__get_plan('s')
dlen = len(data)
prefix, rest = planned[:dlen], planned[dlen:]
if prefix != data:
raise InvalidData('Expected "' + str(prefix)+ '", got "' +
str(data) + '"')
if len(rest) > 0:
self.__plan[0] = ('s', rest)
else:
self.__plan.pop(0)
def all_used(self):
"""
Returns if the whole plan was consumed.
"""
return len(self.__plan) == 0
class ParserTests(unittest.TestCase):
"""
Testcases for the Parser class.
"""
def __terminate(self):
creator = FakeCreator([('s', b'T'), ('r', b'')])
parser = Parser(creator)
self.assertEqual(None, parser.terminate())
self.assertTrue(creator.all_used())
return parser
def test_terminate(self):
"""
Test if the command to terminate is correct and it waits for reading the
EOF.
"""
self.__terminate()
def test_terminate_error1(self):