move DNS_RBT_NSEC_* to db.h

these values pertain to whether a node is in the main, nsec, or nsec3 tree of an RBTDB. they need to be moved to a more generic location so they can also be used by QPDB.

(this is in db.h rather than db_p.h because rbt.c needs access to it. technically, that's a layer violation, but it's a long-existing one; refactoring to get rid of it would be a large hassle, and eventually we expect to remove rbt.c anyway.)