Security Information Exchange (SIE)

Come see SIE Batch (our new way to ingest real-time threat intelligence) and our Newly Active Domains real-time feed

Feb. 24-28 Farsight Booth #3338 South

← Farsight Blog

Running Bulk Queries in DNSDB For A File of IP Addresses



I. Introduction

Most DNSDB API users are familiar with using a command line tool such as dnsdbq to query DNSDB, while others may prefer to use the Farsight DNSDB Scout(tm) browser plugin for Chrome or Firefox.

If you use dnsdbq, you know you can easily run simple IP address queries such as:

$ dnsdbq -i
;; record times: 2018-09-27 03:57:22 .. 2019-07-22 14:44:31
;; count: 4417  A

;; record times: 2018-09-27 11:26:14 .. 2019-07-22 15:51:30
;; count: 1538  A

However, sometimes you may have a file with many IP addresses you want to check in DNSDB.

For the purpose of this blog, we'll assume that you're working from a Unix-like system (either a Mac, a Linux box, or a Windows system running the Windows Subsystem for Linux (see )).

II. Using A Shell File To Run Multiple dnsdbq queries

We'll also assume that you've got a file containing a number of IPv4 (and/or potentially IPv6) addresses such as:

You'd like to run those through DNSDB using dnsdbq. You could just edit your list of IP addresses to create a command file consisting of a series of dnsdbq commands such as:

$ cat
dnsdbq -i
dnsdbq -i
dnsdbq -i

You'd then execute that command file by saying:

$ chmod a+rx
$ sh > run-some-ips.output
$ more run-some-ips.output
;; record times: 2018-05-02 11:00:50 .. 2019-07-22 13:22:10
;; count: 5507  A

;; record times: 2018-03-09 05:08:54 .. 2018-11-13 09:35:57
;; count: 16  A

That works, but is somewhat "clunky" and inelegant.

III. The dnsdbq -f option

A more refined option is to use the dnsdbq -f option, which reads in and runs a file of DNSDB queries from <stdin>. That file must be specially formatted (perhaps with your favorite editor) to look like the following for IP queries:

$ cat my-ips.dat

You could then run those queries by simply saying:

$ dnsdbq -f < my-ips.dat > my-ips.output
$ more my-ips.output
;; record times: 2018-05-02 11:00:50 .. 2019-07-22 13:22:10
;; count: 5507  A

;; record times: 2018-03-09 05:08:54 .. 2018-11-13 09:35:57
;; count: 16  A

You're not limited to running only IP queries using the dnsdbq -f option, although the format of the lines in your data file will be a little different.

If you were doing left hand side RRname queries instead of IP queries, your file of queries might look like:


And if you were doing right hand side Rdata name queries, your file of queries might look like:


You can also create an input file that's a mixture of IP queries, RRset name queries, and Rdata name queries, if that happens to be convenient for you – just use the appropriate prefix for each line in the file.

We should also mention that you can also use other common dnsdbq options, such as asking for output in JSON lines format, or using time fencing, or specifying the maximum number of results to return for each query, should you desire to do so.

See the dnsdbq manual page for details about all dnsdbq options:

$ man dnsdbq
dnsdbq(1)                 BSD General Commands Manual                dnsdbq(1)

     dnsdbq -- DNSDB query tool

     dnsdbq [-dfgmhjsShcI] [-A timestamp] [-B timestamp] [-b bailiwick]
            [-i ip] [-l query_limit] [-L output_limit] [-u server_sys]
            [-n name] [-k sort_keys] [-P page] [-p output_type] [-R raw_rdata]
            [-r rdata] [-t rrtype] [-J input_file]

     dnsdbq constructs and issues queries to the Farsight DNSDB and displays
     responses. It is commonly used as a production command line interface to
     the DNSDB API server.

     DNSDB is a database that stores and indexes both the passive DNS data
     available via Farsight Security's Security Information Exchange as well

IV. Quota Considerations

When doing bulk queries, please note that each query (whether a separate dnsdbq query run as part of a script, or a line in a bulk dnsdbq -f run), will consume one query from the user's quota.

If the user is using a free DNSDB Community Edition ("DCE") key, burst rate limitations clamp the number of bulk queries you can do per hour to no more than 25. If you accidentally attempt to do more than 25, you'll get a 429 error status with the text "Error: Burst window rate limit exceeded. (This is an intentional constraint associated with all DCE keys, but which doesn't apply to regular DNSDB API keys)

V. Conclusion

You now know two approaches you can use to easily run a "file full" of queries in DNSDB with dnsdbq.

If you don't currently have a DNSDB API key, but would like to try DNSDB, please see

I would like to thank Farsight Security staff David Waitzman and Jeremy Reed for their feedback on this article.

The Farsight Security Sales Team can be reached at or give them a call at +1-650-489-7919.

Joe St Sauver Ph.D. is a Distinguished Scientist with Farsight Security®, Inc.

← Blog Home