It’s possible to validate a signature with oXygen but all that says is “Invalid Signature” if there’s an error.

So I’ve put together some Java code which produces a bit more diagnostics; see ValidateEnumCreateJava.zip (or as a .jar file if you don’t have a Java compiler)

Before you start

I recommend doing an XML validity check first: don’t waste time trying to debug XML signature problems if you haven’t.

One way to do this is to use Sun’s Multi Schema Validator - https://msv.dev.java.net/ as suggested in the README in our schema bundles, i.e.
java -jar /path/to/msv.jsr /path/to/nom-enum-root-2.0.xsd your_file.xml


Running the ENUM signature checker

• compile as:
  
javac ValidateEnumCreate.java


• run as:
  
java ValidEnumCreate <yourfile>


or if you don’t have a Java compiler…

• run from a .jar file:
  
java -jar ValidEnumCreate.jar <yourfile>


Results:

Valid Signature

If all is well, the result should be

Signature Validated OK

The response for an invalid signature depends on what was wrong:

Bad DigestValue

If the digest is different but the signature of that digest is correct, the result will be

Signature 0 failed core validation:

Checking that the digest matches the data:
FAIL: DigestValue does not match data
    (Signature 0 ref[’0′] validity status: false)

Checking the signature of the digest:
PASS: SignatureValue verifies DigestValue
    (Signature 0 validation status: true)

This is possibly due to munging of whitespace. The signed XML is fragile and even sensitive to changes in whitespace between tags (I commented on this in an earlier blog article)

Bad Signature or certificate

If the signature is invalid or the wrong certificate is included, the results will be:

Signature 0 failed core validation:

Checking that the digest matches the data:
PASS: DigestValue matches data
    (Signature 0 ref[’0′] validity status: true)

Checking the signature of the digest:
FAIL: SignatureValue does not verify DigestValue
    (Signature 0 validation status: false)

Other errors

• Failure to parse the XML - error message
• Failure to decode the Digest/Signature/Certificate - Java exception + stack trace

References

http://jtute.com/java6/0904.html
http://java.sun.com/developer/technicalArticles/xml/dig_signature_api/
http://weblogs.java.net/blog/mullan/archive/2006/01/my_xml_signatur_1.html
http://weblogs.java.net/blog/2007/08/03/even-more-xml-signature-debugging-tips

The graphs (which are in SVG format) split the DNS-related RFCs into three groups (although some RFCs end up in more than one group):

• Core Protocol RFCs
• Resource Record Definitions
• DNS Operational Guidelines

The point of these graphs is not to show which RFCs refer to other RFCs, but to show which RFCs update or obsolete other RFCs. Hence the graphs give an “at a glance” overview of which RFCs define the DNS protocol as it is now.

Boxes in grey indicate obsoleted RFCs, and square boxes indicate Informational or Best Current Practice documents.  Hovering over a box should tell you the title of the RFC, and clicking on a box will take you to the RFC itself.

The picture below is just a low resolution sample - click on the picture or on the links above to access the scalable SVG versions.

Please let me know if you believe I’ve missed anything, or miscategorised any document.

If we look at Nominet’s EPP service, the “EPP server” itself is actually multiple load balanced servers, operating multithreaded processes. These communicate with xml translation hardware also through load balancers, and also communicate with a database. A combination of factors can cause a difference in the sequence that transactions are acted upon. These include:

1. which EPP server the request goes to
2. thread scheduling (at the operating system level) in the specific EPP server
3. which piece of hardware the xml load balancers select
4. scheduling of translation requests within the xml hardware
5. internal scheduling within the database

When you look at how EPP handles requests over “large” periods of time, EPP is clearly a first come, first served system. However because of the nature of multithreaded systems, it is not feasible (or desirable) to apply that principle when the period of time is a handful of milliseconds. Most of the advantages that EPP has over the Automaton stem from the fact that it is multithreaded and acts on requests in parallel.

The principle behind first come first served is that no party is given preferential treatment when registering a domain name. We do not shape our traffic and no registrar is given any sort of priority when our EPP system processes a request. We apply first come first served based on when the first valid request gets committed to the database. We must do this as we operate three different registration systems.

For registrars who work in an environment where milliseconds can mean the difference between successfully registering a domain name or not, this may be significant when deciding how your EPP client communicates with Nominet.

Recently, I have been monitoring the queries coming to our WHOIS service  and have noticed that several requests were originated by machines belonging to the IP space of a well-known commercial cloud. Since the WHOIS is a free service and can be run from any machine, I strongly suspect this technique has been used to avoid hitting the limit of 1000 queries/day set by Nominet’s Acceptable Use Policy on a per user basis (and not per IP).

The impact of this episode, as far as I can see, is limited and, maybe, not worth too much attention. What is interesting, however, is the way the cloud has been used to circumvent Nominet’s rules. This rises questions about how easy it would be for a malicious user to exploit a cloud computing environment for illegal activities and how long shall we wait before the first large-scale attack based on this technology is reported.

If we consider how the cloud environment works, we realise that:

• A cloud gives a malicious user access to a virtually unlimited pool of resources and computing power
• It is difficult to enforce limits on the amount of resources a single user is allowed to control, because this would harm legimitate users, without preventing malicious ones to open multiple accounts
• Monitoring all processes and activities that run on the cloud is quite complex, maybe impractical. Besides, I don’t think legitimate users would be happy with service providers inspecting their data. They will be forced to use cryptography, which will make things even worse
• Assuming that a service provider could offer some level of protection from misuses of their service, malicious users could spread their activities across different cloud providers, making the task of early detection very complex.
• Finally, accessing cloud services is cheap and prices are expected to drop with the technology behind big data centres becoming more accessible.

The security issues associated to cloud computing are not unknown (recently, for example, botnet controllers have been discovered in the Google cloud), the problem is that this kind of attacks and  the threat associated to them are likely to increase in the coming years.

Defending from a cloud-based attack might not be easy and will need to rely on the “good will” of  the cloud service providers, which will be expected to monitor their users activities. And, to cite Joze Nazario, from Arbor Networks in a recent interview to The Register, “going to a company as big as Google and saying ‘Can we get an image of that server,’ that’s a pretty high barrier”. Especially for small-medium organisations affected by a small/medium -sized attacks.

We took a look at Java mutation testing tools some time ago as a possible addition to our continuous integration system.  But at that time the tools fell short.  Jester works by mutating the source files.  We found that this was just too slow with all the compilation it needs to do.  Jumble is a bit smarter in that it mutates the bytecode of the compiled class files.  However, when we evaluated it only JUnit 3 tests were supported and we were already well on the way to transitioning most of our tests to JUnit 4.

Recently though, Jumble was modified to work with JUnit 4 tests, so I thought it was time to take another look at it.  I found it quite tricky to get it working as there is no ant integration and it only runs as a command line application.  However, with a bit of persuasion I managed to get it running over our codebase using ant.  The details of how I did this are given below, but I think that a better solution would definitely be a fully fledged custom ant task.

So how did I get on?  Initially I had some niggly classloader problems. It seems that you need to tell Jumble’s mutating classloader to defer the loading of various sets of classes to the default classloader.  I needed to do this for the JMX classes found under javax.management by specifying the command line flag --defer-class=javax.management. Once I’d done this I had it working and did indeed find some interesting things.  I found tests that had been cut-and-pasted and not changed to test what they claimed to test. I found some test data that wasn’t up to the job and I found an actual bug in the code.

However, I hit a roadblock once the code under test used a database.  For some reason Oracle’s JDBC driver would not behave. Even before any mutations were applied it would insist on trying to connect with a null password.  This meant that it tried a number of times before locking itself out of the database.  I assume that this is some kind of classloader thing, but it seems strange that it manages to successfully contact the database only to completely mess up the credentials.  I’ve contacted the developers of Jumble (who are based in New Zealand incidentally), but no solution has been forthcoming as yet.  Until this problem is fixed we won’t be able to add this to our continuous integration system, which is a shame, as I think it could be a useful tool.

So, how did I integrate Jumble into ant?  I used the follow macrodef to run Jumble:

<macrodef name="do-mutation">
    <attribute name="class-to-mutate"/>
    <sequential>

        <!-- Use this trick to convert a reference to a classpath to classpath as a string -->
        <property name="the-classpath-as-a-string" refid="execute-test-classpath"/>        

        <java dir="${base.directory}" classname="com.reeltwo.jumble.Jumble" fork="true">

            <!-- all we really need in this classpath is the jumble library -->
            <classpath refid="ants-own-classpath"/>

            <!-- but then it needs everything in the JVM it forks: -->
            <arg value="--classpath=${the-classpath-as-a-string}"/>
            <arg value="--exclude=equals,hashCode,toString"/>
            <arg value="--defer-class=javax.management."/>
            <arg value="@{class-to-mutate}" />
        </java>

    </sequential>

</macrodef>

To get this to work you will need the classpath used to run the tests set up with the id execute-test-classpath and the classpath used by ant to find custom tasks set up with the id ants-own-classpath. The latter will need to include the jumble jar. As you can see I have excluded the equals(), hashcode() and toString() methods from being mutated as the first two are often generated by the IDE anyway. In the case of toString, this rarely contains important logic.  As mentioned before, the JMX classes are deferred to the default classloader. You may need to add some other packages to get it to work in your environment.

The above macrodef will mutate a single class. But we want to run this across our whole code base. To do this I had to resort to using some further ant tricks in the shape of the ant-contrib library which adds additional functionality to ant. As I said before, a better solution would be to write a proper ant task. Here is how the macrodef is called to run Jumble over a whole project:

<target name="mutation.test">    

    <!-- The ant contrib jar contains the "for" task -->
    <taskdef resource="net/sf/antcontrib/antlib.xml" classpathref="ants-own-classpath"/>

    <!-- Strip off the leading directory and the .class. Then replace the slashes
         with dots and separate each one with a comma.
         Results go into the classlist property -->

    <pathconvert dirsep="." pathsep="," property="classlist">
        <mapper type="glob" from="${build.dir}/*.class" to="*"/>
        <fileset refid="classes-to-mutate"/>
    </pathconvert>

    <!-- Iterate through the comma separated list and call jumble -->
    <for list="${classlist}" param="class">
        <sequential>
            <do-mutation class-to-mutate="@{class}"/>
        </sequential>
    </for>

</target>

This takes a fileset with id classes-to-mutate which consists of classes under the directory ${build.dir}. It turns the path into a package and removes the .class from the end to get a list of classes to mutate. (NB This was written to work with Unix style paths, it may need alteration to work under Windows). Then the macrodef given previously is called for each. Note that we refer to the ants-own-classpath classpath again which must contain the ant contrib jar this time.  The code given above was put in a standard ant build file included by other projects. The fileset to mutate could then be defined in each like this:

<fileset id="classes-to-mutate" dir="${build.dir}" includes="**/*.class">
        <exclude name="**/WeWantToLeaveThisOut.class"/>
</fileset>

UPDATE: The story gets even weirder. One of the Jumble developers got back to me with a suggestion for how to fix the Oracle problem, which was to add the JDBC driver to the list of classes deferred to the parent classloader. This didn’t help, but I then discovered bizarrely that the JDBC problem goes away if you are connecting to an 11g database as opposed to a 10g one. So it means that somehow the 10g JDBC driver fails to connect to 10g when run by Jumble, but succeeds against a later version. Curiouser and curiouser…

evldns is a software mashup - it takes libevent’s fast event processing code and combines it with ldns’s DNS packet handling.  It’s derived from the server-side half of libevent’s “evdns” component.

The resulting framework is particularly intended for writing servers which generate custom responses. Examples included are:

• an AS112 server which has been benchmarked at over 60,000 queries per second on an HP DL385 server.
• a server which responds with the IP address of the client which sent the query - this can be useful for network discovery

The framework could also be used to write a “fuzzing” DNS server - one that deliberately returns malformed responses so as to trigger and test for bugs in DNS clients.

Here’s an extract from the package’s README:

evldns works using callback functions. A list of packet matching patterns
may be registered, along with a pointer to the function that will be
invoked when each pattern is matched.

The packet match works on the usual DNS triple of (QNAME, QCLASS, QTYPE)
where QNAME may be an exact match or a wildcard, and QCLASS or QTYPE may
be “ANY”.

The callback function is passed two parameters:

void callback(struct evldns_server_request *req, void *data)

The “req” parameter contains the complete received DNS request as an
“ldns_pkt”. The callback should create a response packet and populate
“req” with that response, which may either be in raw wire format
(req->wire_response and req->wire_len) or in ldns format (req->response).

If the callback function fails to populate either of the response fields
then the evldns system will pass the received packet onto the next
matching callback.

Should no callback match then evldns will automatically generate and
return a packet with RCODE = 5 (Refused).

The “data” parameter is used to pass an additional parameter supplied when
the callback function was registered. See “mod_txtrec.c” for an example
of how “data” may be used to pass expected response data into a callback.

A complete evldns application requires just a few lines of code:

event_init(); /* initialise libevent */
evldns_init(); /* initialise evldns */

/* create an evldns server context */
struct evldns_server *server = evldns_add_server();

/* register a UDP socket with evldns */
evldns_add_server_port(server, bind_to_udp4_port(53));

/* register callbacks here */
evldns_add_callback(server, qname, qclass, qtype, callback, data);
...

/* and set libevent running */
event_dispatch();

Please see the project home page for more information. There is also a Google hosted discussion group.

Ray Bellis, Advanced Projects Team

Our experiments suggest that Nominet systems experienced an analogous, although orders of magnitude smaller, phenomenon. The following figures show the number of new registrations per hour of domain names that contain the name of Michael Jackson (or part of it) and the number of WHOIS queries that Nominet systems received in the same period.

The two graphs are highly correlated because it is common practice for domain name owners to make WHOIS lookups around the period of time they register new domains. The peak around the 27 of June in the second graph is probably related to news stories concerning suspicions about Michael’s death.  Apparently, it did not lead to an immediate rise in the number of domain name registrations.

We have conducted an informal analysis of the domain names that were registered in the last week. The majority of them belong to three categories: parking pages, commercial pages and commemorative sites such as blogs and forums. At the moment, we have no evidence of domain names used for scam or phishing.

In general, this episode confirms (again) that the dynamics of the Domain Name System follow those of the “real world”. A question that is still partially unanswered is at which degree these dynamics are followed by Internet users, i.e. how much their navigation behaviour depends on news stories. In the following months we plan to study the correlation between DNS data and other public events. Google has done something similar in the past, by correlating Google searches for flu-related terms with the spread of flu in North America. The results are very interesting and definitely merit extension to other data sources such as DNS traffic.

In March, I presented the paper Typo-Squatting: The “Curse” of Popularity in the poster session of the first International Conference on Web Science in Athens. The paper, written together with co-authors David Duce and Faye Mitchell (Oxford Brookes University) and Stephen Morris (Nominet) can be downloaded here.

In the paper we study typo-squatting from a statistical point of view. The distribution of names in the co.uk registry is analysed using the concepts of syntactic and visual neighbourhoods of a domain name (the sets of all other domain names which are syntactically or visually similar to to it).  Our preliminary results show a strong correlation between the popularity of a domain name and the size of its syntactical and visual neighbourhoods although, counter-intuitively, the neighbourhood size does not depend on length.  This suggests anomalous activity “around” very popular domain names, as well as indicating that the size of the neighbourhood can be used as a reliable indicator for the likelihood of being typo-squatted.

This application adds ENUM (E.164 Number Mapping) support to your Android phone.

Each time you dial a full international number (i.e. starting with a ‘+’) your phone will check the DNS for additional routing information and offer you a list of alternate contact methods.

The application is open source (under the Apache License) and the code is available for download from Google Code.  The application can be downloaded from the Google Market under Applications -> Communication

Here are some screenshots, which show in turn:

1. Nominet’s switchboard number being dialled
2. ENUM results being returned
3. A call being placed over the PSTN to a tel: URI
4. The ENUM application’s settings page

In brief summary the conference highlighted several key points:

• How we should work with the customer at all stages to ensure both designer and customer are working towards the same goals.
• Developing software that is intuitive, aligned with user behaviour, can really make a software product stand out from it’s competitors.
• How using prototypes before starting development can really help iron out usability bugs before investing too much time and expense.
• Designing good interfaces for complex systems is hard!

My own detailed notes can be found in “UXLondon – Notes on User Experience and Design” (with one of the presentations embedded), where I talk more about what I personally learnt and what I thought of the individual sessions attended.