techblog 

We recently released a new tool; here’s how it describes itself :

The Rapid Enumeration Tool (RET) is designed to use DNSSEC NSEC records to enumerate quickly zone data whilst evading detection by systems which might be designed specifically to identify zone enumeration activity.

It does this by using one or more open recursive resolvers to forward queries to the authoritative name servers for the zone. Each resolver is configured with its own `personality’, specifying query rates, query failure/success ratio, proportions of query types, query name decoration, etc. This allows the RET to feed queries to each resolver, that are specifically tailored to match the queries that a resolver might typically send to the authoritative name server.

Unlike other NSEC resource record `walkers’, the RET does not explicitly query for NSEC RRs to walk the zone. Instead, it combines a `walker’ approach with a dictionary attack (combined with a random name generator for more awkward cases). This means that discernible artifacts in the pattern of queries that arrive at the authoritative servers should be minimised.

For anyone who might be interested, the download is here.

We started off by using dnsjava but ran into serious problems with the threading model. We were firing off lots of queries all the time, and it seemed that dnsjava wasn’t really designed for heavier use. Each time a new query was sent, dnsjava would fire up another thread to handle the response (which could take minutes to time out).

Our original design compounded this problem by having two threads for each resolver (one a query sending thread, and the other to fill the token bucket which limited the query rate).

So, with a couple of hundred resolvers all being targetted with a high query rate (some of which would time out), there would eventually be thousands of threads running and the system would grind to a halt.

In order to solve this problem, we first had to fix dnsjava. Rather than modify the library itself, we decided to make an independent extension. This extension extended the Resolver implementation to offer a new NonBlockingResolver, with a ResponseQueue interface to the caller.

Now we could run the DNS library in three threads rather than (potentially) three thousand!

We could also use the new ResponseQueue interface to reduce the number of threads running in our code. Instead of a query-sending thread per resolver, we could now run the RET in three threads : a NameFiller, a QuerySender and a ResponseThread. We now had seven threads in total (one to control the whole thing).
The real work is done by the NameFiller and the ResponseThread, which both make heavy use of the NameManager and HoleManager classes; synchronisation abounds!

The hole and name managers are made particularly complex due to the requirement for the RET to be a simple cross-platform download - there could be no database in use here. Instead, all the data had to be kept in Java Lists - these are not at all efficient for larger data sets. This meant that the RET had to control the number of holes to no more than a few thousand at any point (even when enumerating a zone with millions of domains).

Add to this :

• the requirement for the majority of queried names to be for names which exist in the zone (but have not been queried for recently)
• the fact that a lot of domains are “unhelpful” (they won’t yield an NSEC record on direct querying, but instead need a new hole to be opened after them by the guessing of a name which would come soon after)

and the situation becomes quite difficult!

If the RET were for use in earnest, then the List would be replaced with a database and the tight grip on the number of holes could be loosened. This would make it pretty much indetectable.