Processing Passive DNS With the nmsg C API

← Blog Home



At Farsight, we work with a large amount of Passive DNS data, and we’re always looking for more. While this data powers our flagship DNSDB, we also broadcast the raw Passive DNS data on the Security Information Exchange (SIE) (as channel 202) for those interested in an unfiltered real-time view of worldwide DNS activity.

In this article, we present a simple example program showing how to filter some potentially interesting information from our raw Passive DNS feed, using the Farsight Security-maintained open source C APIs nmsg and wdns. If you haven’t already familiarized yourself with the NMSG C API, please read this article for an introduction to the basics.

A Motivating Example

The DNSSEC suite of standards allows domain owners to sign their DNS information, authoritative DNS servers to serve up this signature information, and DNS clients to verify the signatures to prevent DNS forgery. These standards require DNSSEC transactions to use a standardized DNS extension mechanism (EDNS0). Because Farsight’s raw passive DNS information includes complete queries and responses, we can directly observe EDNS0 use.

We can also directly observe EDNS0 non-compliance, and thus find some name servers which are not yet ready for DNSSEC, and the hostnames for which they are authoritative. We present an example program to extract this information:

Code Part 1, Basic Plumbing and Filtering

The full code for the program, dnsqr_filter, can be found on Farsight Security’s blog-code GitHub page.

main.c is a basic skeleton for filtering the raw passive DNS NMSG payloads available on SIE. It provides a command-line interface to a basic nmsg I/O loop supporting a subset of the nmsg sources and destinations supported by nmsgtool, and evaluates nmsg payloads with a filter function dnsqr_filter() to select which packets are written to the output file.

The filter for this sample program resides in edns_filter.c. It looks for Passive DNS messages containing non-EDNS0 responses to ENDS0 queries, and tells the main loop to output these messages.

Code Part 2, Digging In

The filter code starts by declaring variables to hold return codes, parsed DNS messages, pointers to nmsg data, and the result of our filtering.

dnsqr_filter(nmsg_message_t msg) {
        nmsg_res nres;
        wdns_res wres;

        wdns_message_t response = {0}, query = {0};
        uint8_t *data;
        size_t len;
        int result = 0;

It then fetches the DNS response from the base:dnsqr encoded NMSG, returning if no response is present.

	nres = nmsg_message_get_field(msg, "response", 0, (void **)&data, &len);
	if (nres != nmsg_res_success) return result;

Next, we parse the DNS response using the low-level C library wdns:

	wres = wdns_parse_message(&response, data, len);
	if (wres != wdns_res_success) return result;

If the response did not use EDNS0, response.edns.present will be false. If this is the case, we check if the request used EDNS0, and tell the main loop to save this message if it did:

	if (!dns.edns.present) {
		nres = nmsg_message_get_field(msg, "query", 0,
						(void **)&data, &len);
		if (nres == nmsg_res_success) {
			wres = wdns_parse_message(&query, data, len);
			if (wres == wdns_res_success) 
				result = query.edns.present;

Finally, we clean up our parsed wdns messages and return our result.

	return result;
Building and Running

Once you’ve downloaded the code, and made sure you’ve installed the nmsg and wdns libraries, building and running should be as simple as:

	$ make
	cc  -c main.c
	cc  -c edns_filter.c
	cc  -o dnsqr_edns_filter main.o edns_filter.o -lnmsg -lwdns
	$ dnsqr_edns_filter -C ch202 -w edns_failures.nmsg
	... wait a few seconds ...

And you should have a file full of non-EDNS0 DNS activity, which you can inspect with nmsgtool, manipulate using python scripts and the python NMSG API, or crunch with more elaborate C programs if you’re so inclined.

Running Outside of SIE

If you are not on SIE, you can run this code on your own recursive DNS servers by using nmsgtool to gather passive DNS information. You first need to tell nmsgtool the query source IP addresses or subnets your recursive DNS server will be using:

    $ export DNSQR_RES_ADDRS=","

Then run dnsqr_edns_filter listening on a socket, and feed it with nmsgtool:

    $ dnsqr_edns_filter -l -w edns_failures.nmsg &
    $ nmsgtool -i (interface) -V base -T dnsqr -s

or you can use nmsgtool to save a batch of base:dnsqr messages for later filtering:

    $ nmsgtool -i (interface) -V base -T dnsqr -c (count) -w (file)
    $ dnsqr_edns_filter -r (file) -w (somewhat-more-interesting-file)

Of course, if you have this sort of interesting data to work with, you should consider e-mailing to get your data in front of even more security-minded people!

Chris Mikkelson is a Senior Distributed Systems Engineer for Farsight Security, Inc.

← Blog Home

Protect against cybercriminal activity in real-time.

Request demo

Email: Phone: +1-650-489-7919