techblog 

The day following the death of Michael Jackson, Google published a graph showing that their system were heavily hit by queries related to this news. Details can be found on the Google Official Blog.

Our experiments suggest that Nominet systems experienced an analogous, although orders of magnitude smaller, phenomenon. The following figures show the number of new registrations per hour of domain names that contain the name of Michael Jackson (or part of it) and the number of WHOIS queries that Nominet systems received in the same period.

The two graphs are highly correlated because it is common practice for domain name owners to make WHOIS lookups around the period of time they register new domains. The peak around the 27 of June in the second graph is probably related to news stories concerning suspicions about Michael’s death.  Apparently, it did not lead to an immediate rise in the number of domain name registrations.

We have conducted an informal analysis of the domain names that were registered in the last week. The majority of them belong to three categories: parking pages, commercial pages and commemorative sites such as blogs and forums. At the moment, we have no evidence of domain names used for scam or phishing.

In general, this episode confirms (again) that the dynamics of the Domain Name System follow those of the “real world”. A question that is still partially unanswered is at which degree these dynamics are followed by Internet users, i.e. how much their navigation behaviour depends on news stories. In the following months we plan to study the correlation between DNS data and other public events. Google has done something similar in the past, by correlating Google searches for flu-related terms with the spread of flu in North America. The results are very interesting and definitely merit extension to other data sources such as DNS traffic.

Typo-squatting is the practice of registering a domain name with the intent to confuse it with the name of a trademark or a famous other domain name

In March, I presented the paper Typo-Squatting: The “Curse” of Popularity in the poster session of the first International Conference on Web Science in Athens. The paper, written together with co-authors David Duce and Faye Mitchell (Oxford Brookes University) and Stephen Morris (Nominet) can be downloaded here.

In the paper we study typo-squatting from a statistical point of view. The distribution of names in the co.uk registry is analysed using the concepts of syntactic and visual neighbourhoods of a domain name (the sets of all other domain names which are syntactically or visually similar to to it).  Our preliminary results show a strong correlation between the popularity of a domain name and the size of its syntactical and visual neighbourhoods although, counter-intuitively, the neighbourhood size does not depend on length.  This suggests anomalous activity “around” very popular domain names, as well as indicating that the size of the neighbourhood can be used as a reliable indicator for the likelihood of being typo-squatted.

I’m pleased to announce the release of enumdroid.

This application adds ENUM (E.164 Number Mapping) support to your Android phone.

Each time you dial a full international number (i.e. starting with a ‘+’) your phone will check the DNS for additional routing information and offer you a list of alternate contact methods.

The application is open source (under the Apache License) and the code is available for download from Google Code.  The application can be downloaded from the Google Market under Applications -> Communication

Here are some screenshots, which show in turn:

1. Nominet’s switchboard number being dialled
2. ENUM results being returned
3. A call being placed over the PSTN to a tel: URI
4. The ENUM application’s settings page

Last week I attended the user experience conference UXLondon organised by Clearleft, with a solid day of keynote talks, followed by two days of half-day workshops.

In brief summary the conference highlighted several key points:

• How we should work with the customer at all stages to ensure both designer and customer are working towards the same goals.
• Developing software that is intuitive, aligned with user behaviour, can really make a software product stand out from it’s competitors.
• How using prototypes before starting development can really help iron out usability bugs before investing too much time and expense.
• Designing good interfaces for complex systems is hard!

My own detailed notes can be found in “UXLondon – Notes on User Experience and Design” (with one of the presentations embedded), where I talk more about what I personally learnt and what I thought of the individual sessions attended.

I got bitten by a silly bug in my dnsruby code recently - I thought I’d share it here in case anyone else starts pulling their hair out over this in future!

DNSSEC RRSIG records contain the signatures required to prove that a DNS zone has been correctly signed by an entity which possesses the correct keys for the zone. DNS clients can obtain the correct keys for the zone, then use the RRSIG records to prove that the zone (or record of interest) is correct. Of course, these records shouldn’t last forever - the data records are periodically re-signed (possibly with different keys), and the RRSIGs updated. The RRSIG includes an inception time, and an expiration time, to show the period over which it is valid. To verify a set of records, the DNS client must first produce an array of bytes, the digest of which is taken and used as the signature for the records - the salient data of of the RRSIG record (including the inception and expiration times) is included in this set of bytes.

The code I had written to do this was working fine - I had coded in the examples from the RFCs, and done a lot of work with actual signed zones (reading the data from the authoritative servers, and proving that it was correct). However, when I started to try to do this with real-world zone files, I started noticing that the signatures weren’t verifying. At least, they *sometimes* weren’t verifying. Very odd. Of course, with this kind of work, all you get is a Pass/Fail - no clue as to what is going wrong. I could see that the records were being translated to and from text format correctly - all the data in the RRs was showing as fine. However, when I compared the byte sequences prepared by my DNS client and Net::DNS, I noticed that four bytes were different. It turned out that, of the four byte sequences written to for the RRSIG inception and expiration time, there was a difference of 0xE100 - this works out to 16 hours worth of 3600 seconds. At last, I was onto something!

There are several ways to express time in the presentation format of the RRSIG record - “1234567890″ (seconds since 01/01/1970), or “YYYYMMDDHHMMSS” (e.g. “20090608123435″). Dnsruby worked fine with the first, and even translated the second from presentation format and back to presentation format correctly. However, when read using the following line :

return Time.mktime(year, mon, day, hour, min, sec).to_i

I got a 16 hour offset from the correct time! [When converting this time to/from the text format, the translation worked perfectly - it was only when inspecting the internal epoch time that the difference could be noticed]

I changed this to :

return Time.gm(year, mon, day, hour, min, sec).to_i

And suddenly everything worked just fine.

I’m sure that seasoned Rubyists will sneer at me for my stupidity - it did take me some time to track this one down! So, if you start noticing strange 16 hours offsets in your Ruby code, it’s worth checking your usage of the Time class…

In this post, I’d like to look at how to use Dnsruby with DNSSEC. As before, I’ll run these examples in irb, and assume that you’ve included Dnsruby there.

Dnsruby has DNSSEC support switched on by default. This means it will attempt to validate any DNS responses against its trust anchors. However, by default, no trust anchors are configured - to get dnsruby to validate responses, you must first configure a trust anchor (or DLV repository).

Trust Anchors

DNSSEC works by following a chain of trust from parent zone to child zone. This chain of trust must start somewhere - the “trust anchor”. In a world with a signed root, the root would be the anchor. Delegations to children zones would be signed, all the way down to the domain that is being queried. The querier can then be sure that the signed response is genuine.

Unfortunately, the root is not yet signed - we have many “islands of security”. Each island is signed, but has no chain to it from the root. It is possible to configure dnsruby with the keys for these zones using Dnsruby::Dnssec#add_trust_anchor() - it’s also possible to define an expiration time for each anchor. Dnsruby will then follow the chain of trust from the anchor down to the queried domain in the signed zone.

Managing these trust anchors quickly becomes a headache. You need to have secure means of obtaining and verifying them, and rolling over to new keys as time goes on. Fortunately, there are two mechanisms to help with this : IANA’s TAR and ISC’s DLV repository.

IANA (who manage the root zone) have created a Trust Anchor Repository (ITAR) which can be used until the root is signed. This holds delegation records for the DNSSEC-signed TLDs. It is possible to download this repository and configure dnsruby with the anchors. A method to do this is defined in Dnsruby::Dnssec#load_itar, but it is not currently secure. If you need to use the ITAR securely, you are currently advised to add the trust anchors from the ITAR directly into dnsruby. A secure method will be provided in future releases.

DLV

Even if the root was signed, there will still be some domains in unsigned zones, which wish to benefit from DNSSEC security. For example, signed-zone.unsigned-zone.example.org - there can be no chain of trust from the root to signed-zone. A solution exists for signed-zone : DNSSEC Lookaside Validation (DLV). Here, a DLV repository holds secure delegation records for zones like signed-zone. Instead of following the chain of trust from the root, a validator follows the chain of trust from the closest parent zone known to the DLV repository. Of course, this method involves more validation queries for each application query.

As an example, considering querying for random.example.com - first, the query itself must be made. Then, if unsuccessful, a DLV query for random.example.com.dlv.isc.org must be made, followed by a query for example.com.dlv.isc.org, followed by a query for com.dlv.isc.org. If none of these succeed, then the message cannot be validated. Imagine that a response was received for the com.dlv.isc.org zone : then, the chain of trust could be followed through example.com down to random.example.com. Keys discovered from the DLV repository are cached.

Configuring Trust Anchors

To configure a trust anchor (in this case for the uk-dnssec.nic.uk DNSSEC test zone) :

    trusted_key = Dnsruby::RR.create({:name => "uk-dnssec.nic.uk.",
        :type => Dnsruby::Types.DNSKEY,
        :flags => 257,
        :protocol => 3,
        :algorithm => 5,
        :key=> "AQPJO6LjrCHhzSF9PIVV7YoQ8iE31FXvghx+14E+jsv4uWJR9jLrxMYm sFOGAKWhiis832ISbPTYtF8sxbNVEotgf9eePruAFPIg6ZixG4yMO9XG LXmcKTQ/cVudqkU00V7M0cUzsYrhc4gPH/NKfQJBC5dbBkbIXJkksPLv Fe8lReKYqocYP6Bng1eBTtkA+N+6mSXzCwSApbNysFnm6yfQwtKlr75p m+pd0/Um+uBkR4nJQGYNt0mPuw4QVBu1TfF5mQYIFoDYASLiDQpvNRN3 US0U5DEG9mARulKSSw448urHvOBwT9Gx5qF2NE4H9ySjOdftjpj62kjb Lmc8/v+z"
      })
    Dnsruby::Dnssec.add_trust_anchor(trusted_key)

Dnsruby will now attempt to validate any responses from the uk-dnssec.nic.uk zone (or its children).

To configure dnsruby to use ISC’s DLV repository, you must first obtain the key (from here). You can then configure dnsruby :

    dlv_key = RR.create("DLV_KEY_STRING_FROM_ISC")
    Dnssec.add_dlv_key(dlv_key)

This method queries the DLV registry to get the ZSK (zone signing key) from the above KSK (key signing key). Dnsruby will now attempt to validate all responses against the DLV repository, if it can’t validate from any trust anchors.

Configuring Validation Policy

It is possible to configure the validation policy to vary the precedence of search order - from the root only, or local anchors only, or either first. Separate key caches are maintained by each validator, making it possible to configure them dynamically. DLV validation is only performed once the DLV key has been added. Here is an example of changing the validation policy :

    Dnsruby::Dnssec.validation_policy = Dnsruby::Dnssec::ValidationPolicy::ROOT_THEN_LOCAL_ANCHORS

It is possible to clear all trusted keys (which will also stop DLV validation) by calling :

    Dnsruby::Dnssec.clear_trusted_keys()

You can remove just the trust anchors (leaving DLV keys and validation from the root, and all keys generated from them), and the keys generated from them, by calling :

    Dnsruby::Dnssec.clear_trust_anchors()

Configuring Validation Resolver

When a response is validated, it may be necessary to make several more queries in order to follow the chain of trust. As more queries are made, more chains are followed. Trusted keys are cached as they are discovered (for the length of time they are indicated to be good for) - this means that future queries for domains in those zones will not require so many validation queries to be performed.

It’s possible to configure dnsruby to use different methods for performing the validation queries. They can either be directed to recursive nameservers (which can be the system defaults, or a client-supplied set of addresses), or they can be performed recursively. I have found that many resolvers do not yet speak a perfect dialect of DNSSEC - performing validation queries recursively ensures that the correct DNSSEC-signed responses are received. The default is to perform validation recursively. Of course, while the caches are being built up when dnsruby starts, more queries will be performed than if the queries were directed to recursive nameservers.

To ask dnsruby to use query a recursive nameserver, call :

    Dnsruby::Dnssec.do_validation_with_recursor = false

Dnsruby will now use the system default configured nameservers for validation.

To use a specific set of servers to perform validation :

    res = Dnsruby::Resolver.new({:nameserver => ['192.168.1.1', '192.168.2.1']})
    Dnsruby::Dnssec.default_resolver = res

Validating Responses

Once dnsruby has been configured with a trust anchor, it will attempt to validate any responses for domains within that zone (or its subzones). If it detects that validation is necessary, then it will fire up a new thread to handle that validation. Since many queries may need to be performed in order to validate the reponse, this can take some time longer than the original query would have done alone. This means that the query timing settings in the Resolver class apply only to each query - *not* to the whole validation process.

For example, a query may have a Resolver#query_timeout of 5 seconds. As long as the answer for that query is returned in 5 seconds, then no timeout will occur - even if it then takes another 6 seconds to validate that response. Future versions of dnsruby will include the ability for client applications to receive events detailing the progress of each asynchronous query (e.g. RECEIVED, VALIDATED).

It is possible to disable validation on a Message basis. Simply set :

  msg.do_validation = false

before sending the Message - dnsruby will not validate the response to that query.

Message Security Levels

Messages can have one of four security levels (defined in Dnsruby::Message::SecurityLevel) : BOGUS, UNCHECKED, INSECURE and SECURE. Dnsruby will only raise an error if it detects that a response is BOGUS - this means that the message does not contain the correct set of signatures. INSECURE means that the response has been verified to have come from a non-secured zone. SECURE means that the chain of trust has been correctly followed from a configured trust anchor to the response, and that all signatures check OK. Note that an NXDOMAIN response can still be SECURE - this means that the NSEC(3) records have been verified to prove non-existence.

To check the security level of a Message, use Message#security_level :

  if (msg.security_level == Dnsruby::Message::SecurityLevel::SECURE)
      print "Response was validated OKn"
  end

Examples of Use

DNSSEC examples can be found in the EXAMPLES file in the dnsruby distribution.

Limitations

Dnsruby does not yet perform NSEC3 validation (although NSEC3/NSEC3PARAM records can be read from the wire, or presentation format). This will be added to a future release.

In this post, I’d like to look at how to use dnsruby to accomplish some common tasks.

Getting started

To follow these examples, you’ll need to install dnsruby :

$ gem install dnsruby

I’ll run these examples in Ruby’s interactive shell :

$ irb

First, I need to include Dnsruby :

>> require 'rubygems'
>> require 'dnsruby'
>> include Dnsruby

Now I’ll load the system’s default resolvers :

>> res = Resolver.new

And display them :

>> res.single_resolvers.each {|s| print "Server address : #{s.server}n"}
Server address : 192.168.1.1
Server address : 192.168.2.2

Now I’ll use them to run a couple of queries :

>> ret = res.query("example.com") # Defaults to A record
>> print ret.answer
example.com.	172789	IN	A	208.77.188.166=> nil
>> res.query("example.com", "MX") # Query the MX record

This time, I’ll use some defined nameservers :

>> res = Resolver.new({:nameserver => ["ns1.nic.uk",
        "ns1.nic.uk"]})

Asynchronous Queries

To run an asynchronous query, I’ll define a Queue to hold the results, and then prepare the query. This time, I’ll construct a Message to hold the query data, and set the RD (recursion desired) bit on the header to 0 :

>> queue = Queue.new
>> m = Message.new("co.uk", Types.NS)
>> m.header.rd = false
>> message_id = res.send_async(m, queue, 1)

Now my code can get on with other tasks, until I’m ready to get the response. Queue#pop is a blocking call, but you can check if it is empty using Queue#empty?.

>> id, reply, error = queue.pop # id == message_id

The [id, reply, error] tuple is popped off the queue. The id identifies which query the response is for (it should match the id returned by the send_async call), reply holds the best response that was received, and any errors will be held in error (which should be nil in this example).

Message Options

Now I’ll ask for a Message to be sent without checking (or the response being stored in) the cache. I’ll also make sure that no DNSSEC validation is performed on the response :

>> m.do_caching = false
>> m.do_validation = false
>> res.send_message(m)

I can ask for a Message to be sent without any pre- or post-processing. No EDNS headers are applied, the header flags are not adjusted, and no caching or validation is performed. This method is most useful for tools authors :

>> res.send_plain_message(Message.new("nic.uk"))

TSIG and Dynamic Updates

I can also use TSIG signatures to communicate securely with a resolver. In this example, I’ll use TSIG to sign a dynamic update. First, I’ll have to define the server to use, and the TSIG key to speak to it with :

>> res = Dnsruby::Resolver.new("ns0.validation-test-servers.nominet.org.uk")
>> res.dnssec = false
>> tsig = Dnsruby::RR.create({
        :name        => "rubytsig",
        :type        => "TSIG",
         :key         => "8n6gugn4aJ7MazyNlMccGKH1WxD2B3UvN/O/RA6iBupO2/03u9CTa3Ewz3gBWTSBCH3crY4Kk+tigNdeJBAvrw==",
      })

Now I’ll create the dynamic update packet :

>> update = Dnsruby::Update.new("validation-test-servers.nominet.org.uk")
>> # ... add stuff to the update
>> update.absent("notthere.update.validation-test-servers.nominet.org.uk", 'TXT')

And apply the TSIG signature and send the message :

>> tsig.apply(update)
>> response = res.send_message(update)
>> print "TSIG response was verified? : #{response.verified?}n"

I could also have configured the Resolver to sign *all* packets with TSIG :

>> res.tsig=tsig.name, tsig.key

Recursive Queries

In addition to defining nameservers to do recursive queries on my behalf, I can also get Dnsruby to query recursively from the root. A static cache is built up, so the more client queries that are run, the less packets need be sent per client query.

>> rec = Recursor.new
>> ret = rec.query("uk-dnssec.nic.uk", "NS")

In my next article, I’ll look at how use Dnsruby with DNSSEC.

I’m very pleased to announce the release of dnsruby version 1.30 (RDoc available here). This version constitutes a considerable advance of functionality on previous dnsruby releases. Although it is still possible to use dnsruby in just the same way (which proves useful for tools authors), it is now also capable of full DNSSEC validation, using either ISC’s DLV registry, IANA’s Trust Anchor Repository, or any trust anchors it is configured with.

Clients can configure the validation policy (e.g. whether to try local trust anchors first, followed by ITAR, then DLV, or some other order), whether to use recursion or a local resolver in the validation process, and manage the store of trusted keys themselves. Dnsruby still offers the verify method to verify a Message or RRSet against a set of keys, but also includes the validate method (called automatically by default) which kicks off a separate validation thread (if validation is necessary). This thread follows the chain of trust from the closest trusted key, and records the result in the Dnsruby::Message#security_level. Exceptions are only raised if the SecurityLevel is BOGUS.

Dnsruby::Recursor now includes a static authority cache. This means that dnsruby can now be used to perform entirely recursive querying - especially useful if you don’t trust your local resolvers, or they don’t yet speak a correct dialect of DNSSEC.

Other improvements include dynamic management of configured nameservers - more responsive ones will be preferred over slower, and servers which don’t respond at all, or give broken responses, will be pushed to the back of the line. A cache has been added for received (and validated) Messages. It is possible to avoid use of this cache on a Message basis.

Support for EventMachine has been removed. I’ve had to rework dnsruby’s event system considerably in order to include validation support, and it was no longer feasible to maintain two IO loops. The native Ruby event loop in dnsruby has no known issues, and that is the recommended (and only) option. It should still be possible to work dnsruby into EventMachine projects.

I plan to write another article soon, with more detail on how to achieve common DNS(SEC) tasks with dnsruby. In the meantime, please let me know if you have any issues using the latest version!

In order to register an ENUM, a XML token (RFC5105) is signed by a Validation Agency to say that the user has a given phone number & so has the right to a ENUM registration; this token is sent by an ENUM Registrar to Nominet’s EPP server.

This process of signing a token isn’t entirely straightforward:

The simplest way to sign a token is to use the templatesign tool from the Apache XML Security project.

$ templatesign -r testVA.key password -c -x testVA.cert unsigned_token.xml > signed_token.xml

A token signed by templatesign is valid, as can be checked using checksig (from the same toolkit) or Oxygen.  However when this token is included as part of an EPP Create command, the signature becomes invalid.  It seems that templatesign doesn’t do the XML canonicalization quite right.

Aside:
To make an XML signature, a digest is taken of the XML data being signed; the data is “canonicalized” before calculating the digest.  It seems that canonicalisation is more-or-less just a standard way of inserting whitespace.  I’m surprised this is how it works because it’s rather fragile, compared to calculating the digest using the element names, attributes etc & ignoring the formatting.

One workaround for this is to run templatesign on the entire Create.  Although this is satisfactory for testing, it does not work in the situation where a Validation Agency has to send a signed token to a Registrar, who then wraps this up into a Create command & sends that to Nominet’s EPP server.

It is possible to use Oxygen to sign a token (example) and this will do the canonicalization in such a way that when enveloped in a Create command, the signature remains valid.  However, Oxygen is an interactive GUI tool and not suited to making an automated system.

Oxygen says that it uses the Java version of Apache XML Security. Looking at the examples that come with the Apache package (specifically GenEnveloped.java) gives us another option:

We’ve taken this example from the Apache library and made the appropriate changes to sign an ENUM token. I’ve packaged some example code which contains:

• EnumTokenSigner.java - signs a token in such a way that it will validate even when enveloped in a Create command.  It takes an unsigned token (as a Java String) for input, signs it and returns the signed token
• EnumTokenCreator.java - is one way to create an unsigned token.  This could be done in any number of ways but using a DOM implementation easily handles absent optional elements.
• EnumTokenSignExample.java - is a simple test harness - it just loads files & calls the two above utility classes.  You will at least have to edit this to set the path names of the keys & token properties.  It does not (for example) do any of the error-checking that you would expect in production code.
• a Makefile

Key formats

Java uses a different key format (DER) than OpenSSL (PEM).  It is necessary to convert a key+certificate to this format before using the Java tools.  OpenSSL can perform the conversion:

$ openssl pkcs8 -topk8 -nocrypt -in yourVA_key.key -inform PEM -out yourVA_key.der -outform DER
$ openssl x509 -in yourVA_cert.pem -inform PEM -out yourVA_cert.der -outform DER

I attended the 2009 Hotsos Symposium, an excellent Oracle database performance tuning conference, in Dallas. The event was a great opportunity to hear world-renowned Oracle performance experts present.

One of the most interesting talks was “The Helsinki Declaration: A set of Principles for the IT Community regarding Application Development” by Toon Koppelaars.

Toon described the expansion of features in the Oracle database over the years. He went on to explain that since the advent of Java, more and more functionality has been implemented outside the database. However new frameworks, methods and languages are appearing frequently and often disappearing quickly, sometimes within a couple of years. Many developers are constantly chasing the latest technology because it’s cool and will allegedly solve all presently-experienced problems. This leads to code quickly becoming legacy, having to be re-written and/or no developers having the necessary skills to maintain it. For example how happy or able would your Java developers be to maintain a system built using Struts, a relatively young framework, but now commonly seen as legacy. Would they first spend ages rewriting it, these days called refactoring, to use Spring, the effort for which gives no value to the user.

Although these technologies are changing, what users want has not changed; they still largely want “window on data” applications.

While this is happening the database technology is remaining stable and under-utilised.

Toon recommends replacing this traditional architecture. He has successfully deployed systems using a new architecture, named The Helsinki Declaration (that’s where it was first proposed). This architecture has a thin user-interface layer, deployed in whatever technology/framework is flavour of the month, and business logic and data logic layers implemented in the comparatively very stable database. Only the thin user interface is then vulnerable to the latest fad.

This is described well on his blog. I recommend starting with his first observation and then proceeding to the second, third and fourth observations.

Talking with conference attendees afterwards I was surprised (or maybe I shouldn’t have been) by how many had experienced exactly the issues Toon described on systems development and maintenance projects.