Draft: Resolve "Coverity Scan issues after DNSRPS test library integration"

The original fixes in librpz_lib_open were incorrect. The handle needs to be closed if it is not returned.

Closes #3973 (closed)