techblog 

In order to register an ENUM, a XML token (RFC5105) is signed by a Validation Agency to say that the user has a given phone number & so has the right to a ENUM registration; this token is sent by an ENUM Registrar to Nominet’s EPP server.

This process of signing a token isn’t entirely straightforward:

The simplest way to sign a token is to use the templatesign tool from the Apache XML Security project.

$ templatesign -r testVA.key password -c -x testVA.cert unsigned_token.xml > signed_token.xml

A token signed by templatesign is valid, as can be checked using checksig (from the same toolkit) or Oxygen.  However when this token is included as part of an EPP Create command, the signature becomes invalid.  It seems that templatesign doesn’t do the XML canonicalization quite right.

Aside:
To make an XML signature, a digest is taken of the XML data being signed; the data is “canonicalized” before calculating the digest.  It seems that canonicalisation is more-or-less just a standard way of inserting whitespace.  I’m surprised this is how it works because it’s rather fragile, compared to calculating the digest using the element names, attributes etc & ignoring the formatting.

One workaround for this is to run templatesign on the entire Create.  Although this is satisfactory for testing, it does not work in the situation where a Validation Agency has to send a signed token to a Registrar, who then wraps this up into a Create command & sends that to Nominet’s EPP server.

It is possible to use Oxygen to sign a token (example) and this will do the canonicalization in such a way that when enveloped in a Create command, the signature remains valid.  However, Oxygen is an interactive GUI tool and not suited to making an automated system.

Oxygen says that it uses the Java version of Apache XML Security. Looking at the examples that come with the Apache package (specifically GenEnveloped.java) gives us another option:

We’ve taken this example from the Apache library and made the appropriate changes to sign an ENUM token. I’ve packaged some example code which contains:

• EnumTokenSigner.java - signs a token in such a way that it will validate even when enveloped in a Create command.  It takes an unsigned token (as a Java String) for input, signs it and returns the signed token
• EnumTokenCreator.java - is one way to create an unsigned token.  This could be done in any number of ways but using a DOM implementation easily handles absent optional elements.
• EnumTokenSignExample.java - is a simple test harness - it just loads files & calls the two above utility classes.  You will at least have to edit this to set the path names of the keys & token properties.  It does not (for example) do any of the error-checking that you would expect in production code.
• a Makefile

Key formats

Java uses a different key format (DER) than OpenSSL (PEM).  It is necessary to convert a key+certificate to this format before using the Java tools.  OpenSSL can perform the conversion:

$ openssl pkcs8 -topk8 -nocrypt -in yourVA_key.key -inform PEM -out yourVA_key.der -outform DER
$ openssl x509 -in yourVA_cert.pem -inform PEM -out yourVA_cert.der -outform DER

I attended the 2009 Hotsos Symposium, an excellent Oracle database performance tuning conference, in Dallas. The event was a great opportunity to hear world-renowned Oracle performance experts present.

One of the most interesting talks was “The Helsinki Declaration: A set of Principles for the IT Community regarding Application Development” by Toon Koppelaars.

Toon described the expansion of features in the Oracle database over the years. He went on to explain that since the advent of Java, more and more functionality has been implemented outside the database. However new frameworks, methods and languages are appearing frequently and often disappearing quickly, sometimes within a couple of years. Many developers are constantly chasing the latest technology because it’s cool and will allegedly solve all presently-experienced problems. This leads to code quickly becoming legacy, having to be re-written and/or no developers having the necessary skills to maintain it. For example how happy or able would your Java developers be to maintain a system built using Struts, a relatively young framework, but now commonly seen as legacy. Would they first spend ages rewriting it, these days called refactoring, to use Spring, the effort for which gives no value to the user.

Although these technologies are changing, what users want has not changed; they still largely want “window on data” applications.

While this is happening the database technology is remaining stable and under-utilised.

Toon recommends replacing this traditional architecture. He has successfully deployed systems using a new architecture, named The Helsinki Declaration (that’s where it was first proposed). This architecture has a thin user-interface layer, deployed in whatever technology/framework is flavour of the month, and business logic and data logic layers implemented in the comparatively very stable database. Only the thin user interface is then vulnerable to the latest fad.

This is described well on his blog. I recommend starting with his first observation and then proceeding to the second, third and fourth observations.

Talking with conference attendees afterwards I was surprised (or maybe I shouldn’t have been) by how many had experienced exactly the issues Toon described on systems development and maintenance projects.

Yubikeys are pretty neat! They provide a simple, open and convenient package for generating one time passwords. The price is also at a point where it becomes cheap enough to add a reasonably good layer of additional security for two factor authentication.

The basic idea is to plug in the USB key and press its button. The key registers itself as a USB keyboard, so no special software is required, and the one time passwords are generated in about a second. The key and the authentication server share an AES key and the one time passwords are validated by keeping track of how many times the key has been used.

Quite a lot has already been written about Yubikeys but it did take a bit of searching to find the finer details. The following two links were the most helpful for understanding the implementation and being able to write my own.

1. Yubikey password format and test vectors
2. Yubikey simulator

See what’s inside a Yubikey generated password after the jump…
Continue Reading »

This follows on from my previous posting about Day 1. Unfortunately I was unable to attend the third day, so this is the final installment.

Web as a Platform - Dion Hinchcliffe

This keynote was about the changing landscape of application development and how cloud computing affects it.  The main point of interest was that even though cloud computing may mean vendor lock-in rearing its ugly head, the cost advantages are going to make it very difficult to ignore. He has much of the same stuff on his blog at ZDNet.

What I’ve learnt since the book - Eric Evans

In this talk Eric Evans discussed what he has learnt about domain driven design since he wrote the book on the subject.

He began by  talking about what is essential in DDD.  Collaboration with domain experts to explore and experiment is important - don’t frighten them away by going through an enormous spec line by line.  Then evolve the model and language going forward, establish context boundaries and focus on the core domains.

He said that the building blocks (services, entities etc) are less important than the book might suggest.  But he also said that “Domain Event” is a missing building block.  It can lead to clearer models by representing the state of entities (e.g. a bank account balance based on the transactions on that account) and can also decouple systems (e.g. a reporting service separate from a transaction service).

Aggregates are consistency boundaries first and foremost and should represent a conceptual whole.  You need to take care putting aggregate level properties and invariants on the root.  Sometimes you might need to make a special aggregate object or add a method to the repository.

Something that was in the book that he thought could be left out was Large Scale Structure, which doesn’t come up too often. On the other hand, he suggested a couple of new context patterns:

• Partners - Mutually Dependent and Cooperative
• Big Ball of Mud - Common.  Know that you can’t do sophisticated modelling within and accept that.

On that second pattern, he said we should accept that not all a large system will be well designed and that precision designs are fragile. It is also possible to spread modelling too thin. Ensure the core domain has a clean context boundary and focus your efforts there. To do the context mapping follow these steps (it isn’t a creative activity):

1. What models do we know of? Draw a blob and a name for each.
2. Where does each apply?
3. Where is information exchanged?
4. Which pattern maps the relationship?

Finally he talked about DDD and SOA, which is something he is often asked about.  He thinks the confusion arises because SOA services carry much more baggage than DDD services, which are just something that can do something for you.  SOA services are defined within some context and it is quite possible that the service interface may define a context boundary.

OSGi - Neil Bartlett

This was an introductory talk on modularisation using OSGi.  He started by saying that while jars have dependencies, these are implicit.  OSGi uses modules, which are just jars plus dependency metadata.  He had a tongue-in-cheek dig at SOA by pointing out that since we can’t use classes in isolation due to the dependencies we have to call them remotely instead. I was surprised to find out that Spring-OSGi had to be renamed Spring-DM (”Dynamic Modules”) due to pressure from the OSGi Alliance. The takeaway message was that you need to use something like Spring-DM to take care of OSGi for you and don’t code directly to services.

Coaching Self-Organising Teams - Joseph Pelrine

This session used cooking soup as a metaphor for coaching an agile team.  So you need to keep things cooking nicely without burning.  I wasn’t quite sure what to take from this.

Value Objects in Domain Driven Design - Dan Bergh Johnsson

This was an entertaining talk on value objects in DDD, which seem to be the poor relation compared to entities.  Lots of this was just good old fashioned object oriented design, with a domain driven slant. Value Objects differ from Data Transfer Objects since DTOs are a technical construct, whereas VOs are true domain objects.

He showed how pulling Value objects out simplifies the code and helps to avoid bugs, awkwardness and duplication.  It also helps the business logic concentrate on just that, without getting bogged down in other issues.  He worked through some examples, with his credit card exchange rate example particularly compelling.  Having orthogonal classes means that in general m+n tests will be needed where previously m*n would be required.

Good candidates for refactoring include strings or integers with limitations and return values from service methods.  For example, you could encapsulate a quanitity so that it is never negative.  Apparently a friend of his working for “a large Swedish corporation” found that anyone entering negative quantities in their online shop would get sent a credit note!

He moved on to discuss how encapsulating multi-object behaviour with composite value objects can improve matters further.  So you might have DateFrom+DateTo=TimeInterval or Amount+Currency=Money or CurrencyFrom+CurrencyTo+BigDecimal=ExchangeRate. He mentioned Qi4j, which is a framework designed to help do this.

I was lucky enough to be able to go along to QCon London 2009, but not quite lucky enough to go to all three days.  So I’ll summarize my thoughts from the two days I did attend.  Here are my notes from Day 1:

Tony Hoare - Opening Keynote

This was a talk about the differences between Computer Scientists and Software Engineers.  He characterised the former as being idealists who think about generalities and the long term, while the latter are pragmatists and specialists who generally consider the short term.  It was a philosophical talk and since I fall into the latter camp I would have liked something more concrete to take away.  He seemed to be very optimistic about what Computer Science can offer the practicising engineer.  In fact, he came out with the quote:

One day, software will be the most reliable component in any system that contains it.

It wasn’t clear quite how we’ll get to this point, although a colleague of mine quipped that this would only happen because less reliable components would be introduced!

David Pollak - Scala and Lift

This was an interview with the maker of the Lift Framework, which is a web framework based on Scala. It was filmed and should appear in due course on InfoQ (who were co-organisers of the conference).  His pitch for this framework was similar in many ways to that of Rails, in that Java frameworks tend to suffer from too much boilerplate code, because it is not possible to easily pass blocks of code around.  In Ruby and Scala this is possible and so this sort of duplication can be removed.  In fact, Lift started out as a port of Rails to Scala.  At least he didn’t call it Scails!

Scala allows you to use type inference to keep the quantity of code down.  But where you are keen to document what types are needed, you can do this too.  He said that research has shown that the number of lines of code written per day varies very little between languages.  This means we should use more expressive languages where possible.

Lift itself uses a more event-driven style of development to avoid the HTTP request/response cycle.  He described it as being more like Visual Basic where you add a button and then add the code to handle the button being pressed.

Programming in the Small - Ivan Moore and Mike Hill

This was an excellent session.  It turned into something like a 60-way refactoring session.  They gave out samples of code and then asked for ideas on how to improve it.  Their premise, which is similar to that in the book Clean Code, is that you need to start out by cleaning up the really small stuff.  Once that is sorted, bigger issues will become more obvious and easier to solve.

To begin, remove useless comments and unneeded code, fix formatting and rename variables to explain their purpose.  One top tip was to check in reformatting as a revision by itself so as not to hide real changes.  The next step was to remove duplication, simplify conditionals and use tiny policy objects if need be to achieve this.  Finally, look to remove statics and NoJos and  hide primitives inside domain objects where possible.

Clojure - Rich Hickey

Clojure is a dialect of Lisp that runs on the JVM and this talk was given by its creator.  He said that it was the interaction between your code and the compiler that makes a Lisp a Lisp.  He did try to convince us that Lisp does not have a problem with too many parentheses, but it turned out he’d tried to solve this (non-)problem in Clojure, so I’m not sure.  He talked about the use of functional languages and how they simplify programming for our new multi-core world.  Clojure solves this with mutable references to immutable objects.  He gave various reasons why he saw object orientation as a problem and said that polymorphism is great, but should be available based on the runtime state of an object,  because “You are not born a father”.

 Three Years of Real-World Ruby - Martin Fowler

This talk looked at the experience of ThoughtWorks using Ruby in real projects.  Generally it seemed to be a success and even those where it was not, the issues were more sociological than technological.  It also seemed that having some experienced developers who knew about dynamic languages was important.  Once a small team had put the foundations in place it would be possible to expand the team. He suggested that this idea (start small and grow out) would be more widely applicable.

Some of the issues traditionally associated with Ruby were mentioned.  He said that so-called Monkey Patching could be controlled by putting all such changes in one place, or by putting the new code into a module to include into the class. On the performance front, he said that Ruby is indeed slow, but that most of the time your application is going to be database bound anyway.

He finished off by talking about the difficulties of testing code based on Rails’ ActiveRecord. Two big projects went different directions with this. One decided to just go with the flow and let the tests talk to the database, whereas another decided to use dynamic mocking to avoid this. It turned out that the mocking caused the tests to become brittle, so that project is now moving towards using the database instead.

 Continuous Integration from the trenches - Julian Simpson

This talk began with the observation that continuous integration is a human activity.  It is not sufficient to install CruiseControl, TeamCity or whatever on a server.  If developers are not checking in frequently, it is not happening.

There was discussion of how to signal a broken build and how to label builds.  He pointed out that there is a danger using subversion revision numbers as build labels, as these can be fragile when repository maintenance is carried out.  Deployable builds should be labelled, but big binary files like this should not be kept in version control. An ivy repository is an alternative.

He talked about deployment scripts and how these should be kept under version control.  He gave the example of a bash script used to deploy a system which be put under version control and ‘tested’ but simply running it and checking the exit value.  This actually picked up an error introduced into the script by mistake.  It is also a good idea to have the CI server grab its own configuration from version control.

He then moved onto functional tests.  He suggested starting with smoke tests, but warned that building fully automated acceptance tests was harder than you might think.  He then moved on to talk about running static analysis and other metrics against the code and speeding up feedback by parallelizing the builds or by having a build pipeline.  But you need to be careful.

I’ve recently been struggling to check out a large subversion repository on my Mac.  Each time I tried I’d get an error like:

svn: Can't open file '..../.svn/tmp/some_path/some_file.svn-base': No such file or directory

I had no such problems checking out the same repository on a Linux system. This was somewhat puzzling.

Finally, the answer dawned on me. The default HFS file system on Mac OS X is case insensitive, and an examination of the repository with a web browser showed the problem - there were two files that had the same name, but one had mixed case, and the other was lower case. When checking out the repository, both files are dropped into a temporary directory, but one overwrites the other. When SVN tries to move the temporary files, one of them no longer exists!

To prove the theory, I created a case-sensitive .dmg file with Disk Utility, and was able to check out the whole repository into that virtual disk without any errors. I’m considering whether I should convert my whole disk to case-sensitive mode, although some Google search results suggest that some applications don’t like that either!

In what could be seen as further proof of the imperfection of multi-threaded programming, I have just released another version of dnsjnio. This version includes fixes from Allan O’Driscoll that deal with very obscure synchronisation issues.

It also includes changes to the test system which cope with the fact that Java DatagramSockets have slight differences of behaviour on different platforms. Regardless of the operating system you use, and the power of your computer, you should now have no problems with the dnsjnio test code. Previously, some versions of Linux (and Solaris), and slower machines, could have some issues.

If you’re using dnsjnio in a very high volume environment, I’d recommend upgrading to the latest version.