techblog 

I’ve just released version 1.1 of Dnsruby.

This release adds DNSSEC support. More precisely, this version implements a non-validating stub resolver : the DO bit (in the EDNS header) is set to indicate that DNSSEC is understood. However, the checking is expected to be performed on the remote resolver - therefore, this should be a trusted resolver on a secure link.

If an untrusted resolver or an insecure link is used, then Dnsruby can still be used to verify responses for zones for which the client application knows a trusted key. A cache of trusted keys is maintained by Dnsruby, with new keys (from signed DNSKEY or DS RRSets) added to the cache. However, it is up to the client application to ensure that appropriate queries have been issued to follow the chain of trust from the known trusted key to the key required to verify particular records in the zone. For more details, see the DNSSEC file in the Dnsruby distribution.

Of course, once we have a signed root, then Dnsruby should be changed to automatically follow the chain of trust from the root down to the zone of interest. At that point, I’ll also need to work out a better API that conveys more sensible information about the resolution. The current system of “success/failure” doesn’t handle the many different types of “failure” that can occur in a DNSSEC world.

We recently hit a problem caused by a database row lock. The actual update being made was not particularly important, as it was just a denormalized value pre-calculated for convenience. However, the consequences of the lock were rather nasty, with many processes waiting for the lock to be released. As a result I decided to change the code to allow the update to be deferred. So if the update encountered a lock, it would write its change to a queue table instead, to be applied later. I did this by opening a cursor with FOR UPDATE NOWAIT. This will throw an exception if it can’t get an immediate lock on the relevant rows.

So far so good, but how do you test something like this? The code in question has been around quite a while and is in PL/SQL (these days most new code we write is in Java). This means we use utPLSQL to test this functionality and so the tests themselves are also written in PL/SQL. The answer is the use of autonomous transactions. These allow you to mark a PL/SQL program unit with a special marker PRAGMA AUTONOMOUS_TRANSACTION that makes it run in its own transaction. So if the code we are testing is in the Make_The_Update procedure of the MyPackage package, the test code would look something like this:

.....

    --This Procedure does its thing in another
    --transaction to force an error to occur
    PROCEDURE Update_Externally IS
      PRAGMA AUTONOMOUS_TRANSACTION;
    BEGIN
       MyPackage.Make_The_Update(1000);
       COMMIT;
    END;

    PROCEDURE ut_check_locking IS
    BEGIN

      --Make an update in the first transaction, but don't commit
      MyPackage.Make_The_Update(10);
      --Make an update in the second transaction
      Update_Externally;
      --Check that it wrote to the queue table here
      ......

    END;

.....

For those unfamiliar with utPLSQL, test procedures are named ut_*. The other procedure is just there for convenience. I have left out much of the boilerplate, such as the setup and teardown code and so on.

So let’s look at what it does. The test procedure ut_check_locking makes an update, but doesn’t commit, then calls the other procedure to attempt an update in a different transaction. (Note that if our code wasn’t designed to deal with this situation, this would be a surefire way to generate a deadlock. Luckily Oracle detects these quite quickly, so if you make a mistake you’ll just get an exception after a few seconds). Finally, we check if the relevant rows have been written to our queue table. Doing this in utPLSQL is left as an exercise for the reader.

I initially had some problems with this, because I forgot that any test data needs to be visible from both transactions. Initially the main transaction just created it and then rolled it back at the end. This gave me some odd ORA-01410: Invalid ROWID errors as I was trying to update non-existent rows. Then I realised that I needed to explicitly commit this data in the setup.

I’ve used autonomous transactions in production code for doing things such as logging when you want the main transaction to roll back. But I’ve not seen them used as an aid to test issues around concurrent data access. But they are certainly helpful in that situation too.

Here at Nominet, we have two interfaces to the registry database:

• The automaton, an email system which uses PGP signed lists of key-value pairs (KVPs), e.g:

  operation: domain request
  key: auto-example.co.uk
  account-id: 1000001
  dns0: ns0.auto-example.co.uk 96.152.146.245
  dns1: ns1.auto-example.co.uk 96.152.146.246 1234::1234 

• and EPP, an XML based system. The above request would essentially be written in XML as:

  <epp>
    <create>
       <domain:create>
           <domain:name>auto-example.co.uk</domain:name>
           <domain:account>
               <domain:account-id>1000001</domain:account-id>
           </domain:account>
          <nameservers>
             <ns:create>
                <ns:name>ns0.auto-example.co.uk</ns:name>
                <ns:addr ip="v4">96.152.146.245</ns:addr>
            </ns:create>
            <ns:create>
                <ns:name>ns0.auto-example.co.uk</ns:name>
                <ns:addr ip="v4">96.152.146.246</ns:addr>
                <ns:addr ip="v6">1234::1234</ns:addr>
            </ns:create>
         </nameservers>
      </domain:create>
   </create>
  </epp>

These are two separate processes but basically do the same thing - they write to a database but with a different interface, so we were keen to use as much common code as possible in the system. One way to facilitate this is to use an XSL transformation (using hardware XML acceleration for performance) to convert the XML into KVPs.

Generally, the XSL for this looks like:

<xsl:template match="xml-element-name">key-name:  <xsl:apply-templates/>
	<xsl:text>&#xa;</xsl:text>
</xsl:template>

This tells the XSL processor “whenever you see xml-element-name write out the corresponding key-name and then process the rest of the element”. For leaf elements, processing the rest of the element will get you the XML value added to the output. The 
 adds a linebreak.

So, for example:

<xsl:template match="domain:name">key: <xsl:apply-templates/>
    <xsl:text>&#xa;</xsl:text>
</xsl:template>

converts the

<domain:name>auto-example.co.uk</domain:name>

line in the EPP request into a corresponding KVP:

key:auto-example.co.uk

This is fairly straightforward for most fields in the requests. Dealing with nameserver fields is however a little more complicated as the KVPs need to be enumerated and the name and ip addresses need to be included in the same value. To do this, the XSL needs to output the ‘dns#: ‘ part when it encounters an <ns:create> element and then add the name and ip addresses when it encounters the child elements.

To find where the entry sits in the dns enumeration, it is necessary to use XPath within an <xsl:value-of> element to count the position of the <ns:create> amongst its sibling elements:

<xsl:template match="ns:create">dns<xsl:value-of select="position()-1"/>
    <xsl:text>: </xsl:text>
    <xsl:apply-templates/>
 </xsl:template>

This works ok for nameservers until you consider that there is no restriction on the order of the ip addresses in the XML. The order of different elements in EPP requests is specified by the schema, but <ns:addr> elements are distinguished only by their attributes and there are no restrictions on the order. So, it is possible for an ipv6 address to be provided before the ipv4 address and the resultant KVP in the above example:

dns0: ns0.auto-example.co.uk 1234::1234 96.152.146.246

would confuse the KVP processing code. To get around this, we need to tell the XSL processor to go and find the ip addresses in the correct order when it finds the name. Again, we need to use XPath:

<xsl:template match="ns:name">
  <!-- Add the name-->
  <xsl:apply-templates/>

  <!-- Go find the ip addresses-->
  <xsl:if test=
      "count(following-sibling::*[local-name()='addr' and @ip='v4']) > 0">
    <xsl:text> </xsl:text>
    <xsl:value-of select=
                "following-sibling::*[local-name()='addr' and @ip='v4']"/>
  </xsl:if>

  <xsl:if test=
      "count(following-sibling::*[local-name()='addr' and @ip='v6']) > 0">
     <xsl:text> </xsl:text>
     <xsl:value-of select=
                "following-sibling::*[local-name()='addr' and @ip='v6']"/>
   </xsl:if>
   <xsl:text>&#xa;</xsl:text>
</xsl:template>

We don’t need to worry about the key for the KVP here as this has already been done at the <ns:create> element.
So, to find the ipv4 address, we count the number of following siblings with name *:addr and ip attribute of ‘v4‘. If there is one then add it’s value to the KVP.

Having just upgraded to Leopard (OSX 10.5) I had problems getting my Novatel (aka Merlin) XU870 to connect to the Internet over 3G. I’ve used Vodafone data services for several years now so the settings are burned into my brain:

With Tiger (OSX 10.4) I used to have all that configured, partly in Network Settings and partly in Internet Connect. To connect I would have to dial from Internet Connect.

With Leopard it turns out that Internet Connect is not needed, nor is the phone number. However there is one oddity that is not what you expect.

In the basic Network Settings box you set the Username and Password as above. Then select the advanced button. On that screen you set:

You have to trust me on the model, if you select CDMA then it does not work and the APN is not displayed. It does connect over 3G and does do HSDPA when available. So this is definitely an oddity with the software.

There have been some other blogs about deleting files and adding old drivers, but I already had the most up to date drivers and did not have to touch them. I had a colleague test it who has never installed the drivers at all and it worked fine for him, so I guess this uses the drivers included with Leopard.

Now, in order to connect I just use the WWAN icon on the menu bar (the one that looks like a gradient of vertical bars) and from that menu select Connect. Previously this icon automatically showed dark bars indicating the signal strength, but now it has varying shades of grey until you connect when it turns black. If you want to see some lights to indicate traffic flow then you need to keep the Network Settings screen open.

PS. I tested this, wrote this blog and posted it, all from a train !