Processing Passive DNS With the nmsg C API
By Chris Mikkelson
At Farsight, we work with a large amount of Passive DNS data, and we're always looking for more. While this data powers our flagship DNSDB, we also broadcast the raw Passive DNS data on the Security Information Exchange (SIE) (as channel 202) for those interested in an unfiltered real-time view of worldwide DNS activity.
In this article, we present a simple example program showing how to filter
some potentially interesting information from our raw Passive DNS feed, using
the Farsight Security-maintained open source C APIs
wdns. If you
haven't already familiarized yourself with the NMSG C API, please read
for an introduction to the basics.
A Motivating Example
The DNSSEC suite of standards allows domain owners to sign their DNS
information, authoritative DNS servers to serve up this signature information,
and DNS clients to verify the signatures to prevent DNS forgery. These
standards require DNSSEC transactions to use a standardized DNS extension
EDNS0). Because Farsight's
raw passive DNS information includes complete queries and responses, we can
We can also directly observe
EDNS0 non-compliance, and thus find some name
servers which are not yet ready for DNSSEC, and the hostnames for which they
are authoritative. We present an example program to extract this information:
Code Part 1, Basic Plumbing and Filtering
main.c is a basic skeleton for filtering the raw passive DNS NMSG payloads
available on SIE. It provides a command-line interface to a basic
I/O loop supporting a subset of the
nmsg sources and destinations supported
nmsgtool, and evaluates
payloads with a filter function
dnsqr_filter() to select which packets are
written to the output file.
The filter for this sample program resides in
It looks for Passive DNS messages containing non-
EDNS0 responses to
queries, and tells the main loop to output these messages.
Code Part 2, Digging In
The filter code starts by declaring variables to hold return codes, parsed DNS
messages, pointers to
nmsg data, and the result of our filtering.
It then fetches the DNS response from the
dnsqr encoded NMSG,
returning if no response is present.
Next, we parse the DNS response using the low-level C library
If the response did not use
response.edns.present will be false. If
this is the case, we check if the request used
EDNS0, and tell the main loop
to save this message if it did:
Finally, we clean up our parsed
wdns messages and return our result.
Building and Running
Once you've downloaded the code, and made sure you've installed the
wdns libraries, building and running should be as simple as:
$ make cc -c main.c cc -c edns_filter.c cc -o dnsqr_edns_filter main.o edns_filter.o -lnmsg -lwdns $ dnsqr_edns_filter -C ch202 -w edns_failures.nmsg ... wait a few seconds ... ^C
And you should have a file full of non-
EDNS0 DNS activity, which you can
python scripts and the
python NMSG API,
or crunch with more elaborate C programs if you're so inclined.
Running Outside of SIE
If you are not on SIE, you can run this code on your own recursive DNS servers
nmsgtool to gather passive DNS information. You first need to
nmsgtool the query source IP addresses or subnets your recursive DNS
server will be using:
$ export DNSQR_RES_ADDRS="192.168.0.1, 172.16.1.0/27"
dnsqr_edns_filter listening on a socket, and
feed it with
$ dnsqr_edns_filter -l 127.0.0.1/5353 -w edns_failures.nmsg & $ nmsgtool -i (interface) -V base -T dnsqr -s 127.0.0.1/5353
or you can use
nmsgtool to save a batch of
messages for later filtering:
$ nmsgtool -i (interface) -V base -T dnsqr -c (count) -w (file) $ dnsqr_edns_filter -r (file) -w (somewhat-more-interesting-file)
Of course, if you have this sort of interesting data to work with, you should consider e-mailing email@example.com to get your data in front of even more security-minded people!
Chris Mikkelson is a Senior Distributed Systems Engineer for Farsight Security, Inc.