Commit ddfaf375 authored by Tomek Mrugalski's avatar Tomek Mrugalski 🛰
Browse files

[2246] Merge branch 'master' into trac2246

Conflicts:
	ChangeLog
parents ac0abc3c 12c48d42
6XX. [func] dclink, tomek
7XX. [func] dclink, tomek
libdhcp++: Interface detection implemented for FreeBSD, NetBSD,
OpenBSD, Mac OS X and Solaris 11. Thanks to David Carlier for
contributing a patch.
(Trac #2246, git abcd)
714. [doc] tomek
BIND10 Contributor's Guide added.
(Trac #3109, git 016bfae00460b4f88adbfd07ed26759eb294ef10)
713. [func] tmark
Added DNS update request construction to d2::NameAddTransaction in
b10-dhcp-ddns. The class now generates all DNS update request variations
needed to fulfill it's state machine in compliance with RFC 4703, sections
5.3 and 5.4.
(Trac# 3207, git dceca9554cb9410dd8d12371b68198b797cb6cfb)
712. [func] marcin,dclink
b10-dhcp4: If server fails to open a socket on one interface it
will log a warning and continue to open sockets on other interfaces.
The warning message is communicated from the libdhcp++ via the
error handler function supplied by the DHCPv4 server. Thanks to
David Carlier for providing a patch.
(Trac #2765, git f49c4b8942cdbafb85414a1925ff6ca1d381f498)
711. [func] tmark
Added the initial implementation of the class, NameAddTransaction,
to b10-dhcp-ddns. This class provides the state model logic
described in the DHCP_DDNS design to add or replace forward and
reverse DNS entries for a given FQDN. It does not yet construct
the actual DNS update requests, this will be added under Trac#
3241.
(Trac# 3207, git 8f99da735a9f39d514c40d0a295f751dc8edfbcd)
710. [build] jinmei
Fixed various build time issues for MacOS X 10.9. Those include
some general fixes and improvements:
- (libdns++) masterLoad() functions now use the generic MasterLoader
class as backend, eliminating the restrictions of the previous
versions.
- (libcc) fixed a minor portability bug in the JSON parser. Although
the only known affected system is OS X 10.9 at the moment, that
could potentially cause disruption on other existing and future
systems.
Other notes:
- if built with googletest, gtest 1.7 (and possibly higher) is
required.
- many older versions of Boost don't work. A known workable version
is 1.54.
(Trac #3213, git d4e570f097fe0eb9009b177a4af285cde0c636cc)
709. [bug] marcin
b10-dhcp6: Server crashed when the client sent FQDN option and did
not request FQDN option to be returned.
(Trac #3220, git 0f1ed4205a46eb42ef728ba6b0955c9af384e0be)
708. [bug] dclink,marcin
libdhcpsrv: Fixed a bug in Memfile lease database backend which
caused DHCPv4 server crashes when leases with NULL client id
were present. Thanks to David Carlier for submitting the patch.
(Trac #2940, git a232f3d7d92ebcfb7793dc6b67914299c45c715b)
707. [bug] muks
Using very large numbers (out of bounds) in config values caused
BIND 10 to throw an exception. This has been fixed in a patch
contributed by David Carlier.
(Trac #3114, git 9bd776e36b7f53a6ee2e4d5a2ea79722ba5fe13b)
706. [func] marcin
b10-dhcp4: Server processes the DHCPv4 Client FQDN and Host Name
options sent by a client and generates the response. as a result
of processing, the server generates NameChangeRequests which
represent changes to DNS mappings for a particular lease (addition
or removal of DNS mappings).
Currently all generated NameChangeRequests are dropped. Sending
them to b10-dhcp-ddns will be implemented with the future tickets.
(Trac #3035, git f617e6af8cdf068320d14626ecbe14a73a6da22)
705. [bug] kean
When commands are piped into bindctl, no longer attempt to query the
user name and password if no default user name and password file is
present, or it contains no valid entries.
(Trac #264, git 4921d7de6b5623c7e85d2baf8bc978686877345b)
704. [func] naokikambe
New statistics items related to IP sockets added into b10-xfrin:
open, openfail, close, connfail, conn, senderr, and recverr.
Their values can be obtained by invoking "Stats show Xfrin" via
bindctl while b10-xfrin is running.
(Trac #2300, git 4655c110afa0ec6f5669bf53245bffe6b30ece4b)
703. [bug] kean
A bug in b10-msgq was fixed where it would remove the socket file if
there was an existing copy of b10-msgq running. It now correctly
detects and reports this without removing the socket file.
(Trac #433, git c18a49b0435c656669e6f87ef65d44dc98e0e726)
702. [func] marcin
perfdhcp: support for sending DHCPv6 Renew messages at the specified
rate and measure performance.
(Trac #3183, git 66f2939830926f4337623b159210103b5a8e2434)
701. [bug] tomek
libdhcp++: Incoming DHCPv6 IAPREFIX option is now parsed properly.
(Trac #3211, git ed43618a2c7b2387d76f99a5a4b1a3e05ac70f5e)
700. [func] tomek,marcin
b10-dhcp4,b10-dhcp6: Support for vendor options has been added. It
is now possible to configure vendor options. Server is able to
parse some CableLabs vendor options and send configured vendor
options in response. The support is not complete.
(Trac #3194, git 243ded15bbed0d35e230d00f4e3ee42c3609616c)
699. [bug] marcin
libdhcp++: Options with defined suboptions are now handled properly.
In particular, Relay Agent Info options is now echoed back properly.
(Trac #3102, git 6f6251bbd761809634aa470f36480d046b4d2a20)
698. [bug] muks
A bug was fixed in the interaction between b10-init and b10-msgq
that caused BIND 10 failures after repeated start/stop of
components.
(Trac #3094, git ed672a898d28d6249ff0c96df12384b0aee403c8
697. [func] tmark
Implements "user_check" hooks shared library which supports subnet
selection based upon the contents of a list of known DHCP lease users
(i.e. clients). Adds the following subdirectories to the bind10 src
directory for maintaining hooks shared libraries:
-bind10/src/hooks - base directory for hooks shared libraries
-bind10/src/hooks/dhcp - base directory for all hooks libs pertaining
to DHCP(Kea)
-bind10/src/hooks/dhcp/user_check - directory containing the user_check
hooks library
(Trac #3186, git f36aab92c85498f8511fbbe19fad5e3f787aef68)
696. [func] tomek
b10-dhcp4: It is now possible to specify value of siaddr field
in DHCPv4 responses. It is used to point out to the next
server in the boot process (that typically is TFTP server).
(Trac #3191, git 541922b5300904a5de2eaeddc3666fc4b654ffba)
695. [func] tomek
b10-dhcp6 is now able to listen on global IPv6 unicast addresses.
(Trac #3195, git 72e601f2a57ab70b25d50877c8e49242739d1c9f)
694. [bug] tomek
b10-dhcp6 now handles exceptions better when processing initial
configuration. In particular, errors with socket binding do not
prevent b10-dhcp6 from establishing configuration session anymore.
(Trac #3195, git 72e601f2a57ab70b25d50877c8e49242739d1c9f)
693. [bug] tomek
b10-dhcp6 now handles IPv6 interface enabling correctly.
(Trac #3195, git 72e601f2a57ab70b25d50877c8e49242739d1c9f)
692. [bug] marcin
b10-dhcp4: Fix a bug whereby the Parameter Request List was not parsed
by the server and requested DHCPv4 options were not returned to the
client. Options are not sent back to the client if server failed to
assign a lease.
(Trac #3200, git 50d91e4c069c6de13680bfaaee3c56b68d6e4ab1)
691. [bug] marcin
libdhcp++: Created definitions for standard DHCPv4 options:
tftp-server-name (66) and boot-file-name (67). Also, fixed definition
of DHCPv4 option time-offset (2).
(Trac #3199, git abcd)
(Trac #3199, git 6e171110c4dd9ae3b1be828b9516efc65c33460b)
690. [bug] tomek
b10-dhcp4: Relay Agent Info option is now echoed back in
......
......@@ -21,11 +21,11 @@ dist_doc_DATA = AUTHORS COPYING ChangeLog README
.PHONY: check-valgrind check-valgrind-suppress
install-exec-hook:
-@echo -e "\033[1;33m" # switch to yellow color text
-@tput smso # Start standout mode
@echo "NOTE: BIND 10 does not automatically start DNS services when it is run"
@echo " in its default configuration. Please see the Guide for information"
@echo " on how to configure these services to be started automatically."
-@echo -e "\033[m" # switch back to normal
-@tput rmso # End standout mode
check-valgrind:
if HAVE_VALGRIND
......
......@@ -68,6 +68,13 @@ AC_CHECK_DECL([__SUNPRO_CC], [SUNCXX="yes"], [SUNCXX="no"])
AC_CHECK_DECL([__clang__], [CLANGPP="yes"], [CLANGPP="no"])
AM_CONDITIONAL(USE_CLANGPP, test "X${CLANGPP}" = "Xyes")
dnl Determine if weare using GNU sed
GNU_SED=no
$SED --version 2> /dev/null | grep -q GNU
if test $? -eq 0; then
GNU_SED=yes
fi
# Linker options
# check -R, "-Wl,-R" or -rpath (we share the AX function defined in
......@@ -105,9 +112,12 @@ AC_DEFUN([BIND10_CXX_TRY_FLAG], [
AC_MSG_RESULT([$bind10_cxx_flag])
])
CXX_VERSION="unknown"
# SunStudio compiler requires special compiler options for boost
# (http://blogs.sun.com/sga/entry/boost_mini_howto)
if test "$SUNCXX" = "yes"; then
CXX_VERSION=`$CXX -V 2> /dev/null | head -1`
CXXFLAGS="$CXXFLAGS -library=stlport4 -features=tmplife -features=tmplrefstatic"
MULTITHREADING_FLAG="-mt"
fi
......@@ -120,7 +130,8 @@ fi
# we suppress this particular warning. Note that it doesn't weaken checks
# on the source code.
if test "$CLANGPP" = "yes"; then
B10_CXXFLAGS="$B10_CXXFLAGS -Qunused-arguments"
CXX_VERSION=`$CXX --version 2> /dev/null | head -1`
B10_CXXFLAGS="$B10_CXXFLAGS -Qunused-arguments"
fi
BIND10_CXX_TRY_FLAG([-Wno-missing-field-initializers],
......@@ -129,6 +140,7 @@ AC_SUBST(WARNING_NO_MISSING_FIELD_INITIALIZERS_CFLAG)
# gcc specific settings:
if test "X$GXX" = "Xyes"; then
CXX_VERSION=`$CXX --version 2> /dev/null | head -1`
B10_CXXFLAGS="$B10_CXXFLAGS -Wall -Wextra -Wnon-virtual-dtor -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare"
case "$host" in
*-solaris*)
......@@ -200,6 +212,7 @@ AC_HELP_STRING([--disable-setproctitle-check],
# OS dependent configuration
SET_ENV_LIBRARY_PATH=no
ENV_LIBRARY_PATH=LD_LIBRARY_PATH
bind10_undefined_pthread_behavior=no
case "$host" in
*-solaris*)
......@@ -211,13 +224,25 @@ case "$host" in
# Destroying locked mutexes, condition variables being waited
# on, etc. are undefined behavior on Solaris, so we set it as
# such here.
AC_DEFINE([HAS_UNDEFINED_PTHREAD_BEHAVIOR], [1], [Does this platform have some undefined pthreads behavior?])
bind10_undefined_pthread_behavior=yes
;;
*-apple-darwin*)
# Starting with OSX 10.7 (Lion) we must choose which IPv6 API to use
# (RFC2292 or RFC3542).
CPPFLAGS="$CPPFLAGS -D__APPLE_USE_RFC_3542"
# In OS X 10.9 (and possibly any future versions?) pthread_cond_destroy
# doesn't work as documented, which makes some of unit tests fail.
# Testing a specific system and version is not a good practice, but
# identifying this behavior would be too heavy (running a program
# with multiple threads), so this is a compromise. In general,
# it should be avoided to rely on 'osx_version' unless there's no
# viable alternative.
osx_version=`/usr/bin/sw_vers -productVersion`
if [ test $osx_version = "10.9" ]; then
bind10_undefined_pthread_behavior=yes
fi
# libtool doesn't work perfectly with Darwin: libtool embeds the
# final install path in dynamic libraries and our loadable python
# modules always refer to that path even if it's loaded within the
......@@ -240,6 +265,9 @@ esac
AM_CONDITIONAL(SET_ENV_LIBRARY_PATH, test $SET_ENV_LIBRARY_PATH = yes)
AC_SUBST(SET_ENV_LIBRARY_PATH)
AC_SUBST(ENV_LIBRARY_PATH)
if [ test $bind10_undefined_pthread_behavior = "yes" ]; then
AC_DEFINE([HAS_UNDEFINED_PTHREAD_BEHAVIOR], [1], [Does this platform have some undefined pthreads behavior?])
fi
# Our experiments have shown Solaris 10 has broken support for the
# IPV6_USE_MIN_MTU socket option for getsockopt(); it doesn't return the value
......@@ -718,6 +746,21 @@ then
BOTAN_INCLUDES="-I`${BOTAN_CONFIG} --prefix`/include ${BOTAN_INCLUDES}"
fi
fi
dnl Determine the Botan version
AC_MSG_CHECKING([Botan version])
cat > conftest.cpp << EOF
#include <botan/version.h>
AUTOCONF_BOTAN_VERSION=BOTAN_VERSION_MAJOR . BOTAN_VERSION_MINOR . BOTAN_VERSION_PATCH
EOF
BOTAN_VERSION=`$CPP $CPPFLAGS $BOTAN_INCLUDES conftest.cpp | grep '^AUTOCONF_BOTAN_VERSION=' | $SED -e 's/^AUTOCONF_BOTAN_VERSION=//' -e 's/[[ ]]//g' -e 's/"//g' 2> /dev/null`
if test -z "$BOTAN_VERSION"; then
BOTAN_VERSION="unknown"
fi
$RM -f conftest.cpp
AC_MSG_RESULT([$BOTAN_VERSION])
# botan-config script (and the way we call pkg-config) returns -L and -l
# as one string, but we need them in separate values
BOTAN_LDFLAGS=
......@@ -755,7 +798,24 @@ CPPFLAGS_SAVED=$CPPFLAGS
CPPFLAGS="$BOTAN_INCLUDES $CPPFLAGS"
LIBS_SAVED="$LIBS"
LIBS="$LIBS $BOTAN_LIBS"
AC_CHECK_HEADERS([botan/botan.h],,AC_MSG_ERROR([Missing required header files.]))
# ac_header_preproc is an autoconf symbol (undocumented but stable) that
# is set if the pre-processor phase passes. Thus by adding a custom
# failure handler we can detect the difference between a header not existing
# (or not even passing the pre-processor phase) and a header file resulting
# in compilation failures.
AC_CHECK_HEADERS([botan/botan.h],,[
if test "x$ac_header_preproc" = "xyes"; then
AC_MSG_ERROR([
botan/botan.h was found but is unusable. The most common cause of this problem
is attempting to use an updated C++ compiler with older C++ libraries, such as
the version of Botan that comes with your distribution. If you have updated
your C++ compiler we highly recommend that you use support libraries such as
Boost and Botan that were compiled with the same compiler version.])
else
AC_MSG_ERROR([Missing required header files.])
fi]
)
AC_LINK_IFELSE(
[AC_LANG_PROGRAM([#include <botan/botan.h>
#include <botan/hash.h>
......@@ -797,6 +857,7 @@ if test "$MYSQL_CONFIG" != "" ; then
MYSQL_CPPFLAGS=`$MYSQL_CONFIG --cflags`
MYSQL_LIBS=`$MYSQL_CONFIG --libs`
MYSQL_VERSION=`$MYSQL_CONFIG --version`
AC_SUBST(MYSQL_CPPFLAGS)
AC_SUBST(MYSQL_LIBS)
......@@ -875,6 +936,20 @@ AC_LINK_IFELSE(
AC_MSG_ERROR([Needs log4cplus library])]
)
dnl Determine the log4cplus version, used mainly for config.report.
AC_MSG_CHECKING([log4cplus version])
cat > conftest.cpp << EOF
#include <log4cplus/version.h>
AUTOCONF_LOG4CPLUS_VERSION=LOG4CPLUS_VERSION_STR
EOF
LOG4CPLUS_VERSION=`$CPP $CPPFLAGS conftest.cpp | grep '^AUTOCONF_LOG4CPLUS_VERSION=' | $SED -e 's/^AUTOCONF_LOG4CPLUS_VERSION=//' -e 's/[[ ]]//g' -e 's/"//g' 2> /dev/null`
if test -z "$LOG4CPLUS_VERSION"; then
LOG4CPLUS_VERSION="unknown"
fi
$RM -f conftest.cpp
AC_MSG_RESULT([$LOG4CPLUS_VERSION])
CPPFLAGS=$CPPFLAGS_SAVED
LIBS=$LIBS_SAVED
......@@ -954,6 +1029,7 @@ AC_SUBST(MULTITHREADING_FLAG)
GTEST_LDFLAGS=
GTEST_LDADD=
DISTCHECK_GTEST_CONFIGURE_FLAG=
GTEST_VERSION="unknown"
if test "x$enable_gtest" = "xyes" ; then
......@@ -1009,6 +1085,7 @@ if test "$gtest_path" != "no" ; then
GTEST_INCLUDES=`${GTEST_CONFIG} --cppflags`
GTEST_LDFLAGS=`${GTEST_CONFIG} --ldflags`
GTEST_LDADD=`${GTEST_CONFIG} --libs`
GTEST_VERSION=`${GTEST_CONFIG} --version`
GTEST_FOUND="true"
else
AC_MSG_WARN([Unable to locate Google Test gtest-config.])
......@@ -1288,6 +1365,10 @@ AC_CONFIG_FILES([Makefile
src/bin/usermgr/Makefile
src/bin/usermgr/tests/Makefile
src/bin/tests/Makefile
src/hooks/Makefile
src/hooks/dhcp/Makefile
src/hooks/dhcp/user_chk/Makefile
src/hooks/dhcp/user_chk/tests/Makefile
src/lib/Makefile
src/lib/asiolink/Makefile
src/lib/asiolink/tests/Makefile
......@@ -1468,6 +1549,7 @@ AC_OUTPUT([doc/version.ent
src/bin/d2/spec_config.h.pre
src/bin/d2/tests/test_data_files_config.h
src/bin/tests/process_rename_test.py
src/hooks/dhcp/user_chk/tests/test_data_files_config.h
src/lib/config/tests/data_def_unittests_config.h
src/lib/dhcpsrv/tests/test_libraries.h
src/lib/python/isc/config/tests/config_test
......@@ -1538,39 +1620,69 @@ cat > config.report << END
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Package:
Name: $PACKAGE_NAME
Version: $PACKAGE_VERSION
C++ Compiler: $CXX
Flags:
DEFS: $DEFS
CPPFLAGS: $CPPFLAGS
CXXFLAGS: $CXXFLAGS
LDFLAGS: $LDFLAGS
B10_CXXFLAGS: $B10_CXXFLAGS
OS Family: $OS_TYPE
dnl includes too
Python: ${PYTHON_INCLUDES}
${PYTHON_CXXFLAGS}
${PYTHON_LDFLAGS}
${PYTHON_LIB}
Boost: ${BOOST_INCLUDES}
Botan: ${BOTAN_INCLUDES}
${BOTAN_LDFLAGS}
${BOTAN_LIBS}
Log4cplus: ${LOG4CPLUS_INCLUDES}
${LOG4CPLUS_LIBS}
SQLite: $SQLITE_CFLAGS
$SQLITE_LIBS
Name: ${PACKAGE_NAME}
Version: ${PACKAGE_VERSION}
OS Family: ${OS_TYPE}
Using GNU sed: ${GNU_SED}
C++ Compiler:
CXX: ${CXX}
CXX_VERSION: ${CXX_VERSION}
DEFS: ${DEFS}
CPPFLAGS: ${CPPFLAGS}
CXXFLAGS: ${CXXFLAGS}
LDFLAGS: ${LDFLAGS}
B10_CXXFLAGS: ${B10_CXXFLAGS}
Python:
PYTHON_VERSION: ${PYTHON_VERSION}
PYTHON_INCLUDES: ${PYTHON_INCLUDES}
PYTHON_CXXFLAGS: ${PYTHON_CXXFLAGS}
PYTHON_LDFLAGS: ${PYTHON_LDFLAGS}
PYTHON_LIB: ${PYTHON_LIB}
Boost:
BOOST_VERSION: ${BOOST_VERSION}
BOOST_INCLUDES: ${BOOST_INCLUDES}
Botan:
BOTAN_VERSION: ${BOTAN_VERSION}
BOTAN_INCLUDES: ${BOTAN_INCLUDES}
BOTAN_LDFLAGS: ${BOTAN_LDFLAGS}
BOTAN_LIBS: ${BOTAN_LIBS}
Log4cplus:
LOG4CPLUS_VERSION: ${LOG4CPLUS_VERSION}
LOG4CPLUS_INCLUDES: ${LOG4CPLUS_INCLUDES}
LOG4CPLUS_LIBS: ${LOG4CPLUS_LIBS}
SQLite:
SQLITE_VERSION: ${SQLITE_VERSION}
SQLITE_CFLAGS: ${SQLITE_CFLAGS}
SQLITE_LIBS: ${SQLITE_LIBS}
END
# Avoid confusion on DNS/DHCP and only mention MySQL if it
# were specified on the command line.
if test "$MYSQL_CPPFLAGS" != "" ; then
cat >> config.report << END
MySQL: $MYSQL_CPPFLAGS
$MYSQL_LIBS
MySQL:
MYSQL_VERSION: ${MYSQL_VERSION}
MYSQL_CPPFLAGS: ${MYSQL_CPPFLAGS}
MYSQL_LIBS: ${MYSQL_LIBS}
END
fi
if test "$enable_gtest" != "no"; then
cat >> config.report << END
GTest:
GTEST_VERSION: ${GTEST_VERSION}
GTEST_INCLUDES: ${GTEST_INCLUDES}
GTEST_LDFLAGS: ${GTEST_LDFLAGS}
GTEST_LDADD: ${GTEST_LDADD}
GTEST_SOURCE: ${GTEST_SOURCE}
END
fi
......@@ -1597,4 +1709,8 @@ cat <<EOF
run "make check", you must run "make" first as some files need to be
generated by "make" before "make check" can be run.
When running "make install" do not use any form of parallel or job
server options (such as GNU make's -j option). Doing so may cause
errors.
EOF
......@@ -20,3 +20,237 @@ which algorithm is best for that. If it is very critical, then a
custom algorithm designed for DNS caching makes sense. If it is not,
then we can consider using an STL-based data structure.
Effectiveness of Cache
----------------------
First, I'll try to answer the introductory questions.
In some simplified model, we can express the amount of running time
for answering queries directly from the cache in the total running
time including that used for recursive resolution due to cache miss as
follows:
A = r*Q2*/(r*Q2+ Q1*(1-r))
where
A: amount of time for answering queries from the cache per unit time
(such as sec, 0<=A<=1)
r: cache hit rate (0<=r<=1)
Q1: max qps of the server with 100% cache hit
Q2: max qps of the server with 0% cache hit
Q1 can be measured easily for given data set; measuring Q2 is tricky
in general (it requires many external queries with unreliable
results), but we can still have some not-so-unrealistic numbers
through controlled simulation.
As a data point for these values, see a previous experimental results
of mine:
https://lists.isc.org/pipermail/bind10-dev/2012-July/003628.html
Looking at the "ideal" server implementation (no protocol overhead)
with the set up 90% and 85% cache hit rate with 1 recursion on cache
miss, and with the possible maximum total throughput, we can deduce
Q1 and Q2, which are: 170591qps and 60138qps respectively.
This means, with 90% cache hit rate (r = 0.9), the server would spend
76% of its run time for receiving queries and answering responses
directly from the cache: 0.9*60138/(0.9*60138 + 0.1*170591) = 0.76.
I also ran more realistic experiments: using BIND 9.9.2 and unbound
1.4.19 in the "forward only" mode with crafted query data and the
forwarded server to emulate the situation of 100% and 0% cache hit
rates. I then measured the max response throughput using a
queryperf-like tool. In both cases Q2 is about 28% of Q1 (I'm not
showing specific numbers to avoid unnecessary discussion about
specific performance of existing servers; it's out of scope of this
memo). Using Q2 = 0.28*Q1, above equation with 90% cache hit rate
will be: A = 0.9 * 0.28 / (0.9*0.28 + 0.1) = 0.716. So the server will
spend about 72% of its running time to answer queries directly from
the cache.
Of course, these experimental results are too simplified. First, in
these experiments we assumed only one external query is needed on
cache miss. In general it can be more; however, it may not actually
be too optimistic either: in my another research result:
http://bind10.isc.org/wiki/ResolverPerformanceResearch
In the more detailed analysis using real query sample and tracing what
an actual resolver would do, it looked we'd need about 1.44 to 1.63
external queries per cache miss in average.
Still, of course, the real world cases are not that simple: in reality
we'd need to deal with timeouts, slower remote servers, unexpected
intermediate results, etc. DNSSEC validating resolvers will clearly
need to do more work.
So, in the real world deployment Q2 should be much smaller than Q1.
Here are some specific cases of the relationship between Q1 and Q2 for
given A (assuming r = 0.9):
70%: Q2 = 0.26 * Q1
60%: Q2 = 0.17 * Q1
50%: Q2 = 0.11 * Q1
So, even if "recursive resolution is 10 times heavier" than the cache
only case, we can assume the server spends a half of its run time for
answering queries directly from the cache at the cache hit rate of
90%. I think this is a reasonably safe assumption.
Now, assuming the number of 50% or more, does this suggest we should
highly optimize the cache? Opinions may vary on this point, but I
personally think the answer is yes. I've written an experimental
cache only implementation that employs the idea of fully-rendered
cached data. On one test machine (2.20GHz AMD64, using a single
core), queryperf-like benchmark shows it can handle over 180Kqps,
while BIND 9.9.2 can just handle 41K qps. The experimental
implementation skips some necessary features for a production server,
and cache management itself is always inevitable bottleneck, so the
production version wouldn't be that fast, but it still suggests it may
not be very difficult to reach over 100Kqps in production environment
including recursive resolution overhead.
Cache Types
-----------
1. Record cache
Conceptually, any recursive resolver (with cache) implementation would
have cache for RRs (or RRsets in the modern version of protocol) given
in responses to its external queries. In BIND 9, it's called the
"cached DB", using an in-memory rbt-like tree. unbound calls it
"rrset cache", which is implemented as a hash table.
2. Delegation cache
Recursive server implementations would also have cache to determine
the deepest zone cut for a given query name in the recursion process.
Neither BIND 9 nor unbound has a separate cache for this purpose;
basically they try to find an NR RRset from the "record cache" whose
owner name best matches the given query name.
3. Remote server cache
In addition, a recursive server implementation may maintain a cache
for information of remote authoritative servers. Both BIND 9 and
unbound conceptually have this type of cache, although there are some
non-negligible differences in details. BIND 9's implementation of
this cache is called ADB. Its a hash table whose key is domain name,
and each entry stores corresponding IPv6/v4 addresses; another data
structure for each address stores averaged RTT for the address,
lameness information, EDNS availability, etc. unbound's
implementation is called "infrastructure cache". It's a hash table
keyed with IP addresses whose entries store similar information as
that in BIND 9's per address ADB entry. In unbound a remote server's
address must be determined by looking up the record cache (rrset cache
in unbound terminology); unlike BIND 9's ADB, there's no direct
shortcut from a server's domain name to IP addresses.
4. Full response cache
unbound has an additional cache layer, called the "message cache".
It's a hash table whose hash key is query parameter (essentially qname
and type) and entry is a sequence to record (rrset) cache entries.
This sequence constructs a complete response to the corresponding
query, so it would help optimize building a response message skipping
the record cache for each section (answer/authority/additional) of the
response message. PowerDNS recursor has (seemingly) the same concept
called "packet cache" (but I don't know its implementation details
very much).
BIND 9 doesn't have this type of cache; it always looks into the
record cache to build a complete response to a given query.
Miscellaneous General Requirements
----------------------------------
- Minimize contention between threads (if threaded)
- Cache purge policy: normally only a very small part of cached DNS
information will be reused, and those reused are very heavily
reused. So LRU-like algorithm should generally work well, but we'll
also need to honor DNS TTL.
Random Ideas for BIND 10
------------------------
Below are specific random ideas for BIND 10. Some are based on
experimental results with reasonably realistic data; some others are
mostly a guess.
1. Fully rendered response cache
Some real world query samples show that a very small portion of entire
queries are very popular and queried very often and many times; the
rest is rarely reused, if any. Two different data sets show top
10,000 queries would cover around 80% of total queries, regardless
of the size of the total queries. This suggests an idea of having a
small, highly optimized full response cache.