techblog 

As mentioned by Miquel in a previous article, we are using Coverity Prevent to look for possible defects in our code.  Today I realised that it had spotted a possible problem that is almost invisible to the naked eye.  Initially it flagged up some code that looked like this:

if (someObject.getErrorType() == SomeClass.MY_ERROR_TYPE) {
.....
}

The complaint was that getErrorType() could return a null value, so this comparison would throw an exception when the null pointer was dereferenced. That’s an easy one to fix I thought. I changed the code to look like this instead:

if (SomeClass.MY_ERROR_TYPE == someObject.getErrorType()) {
.....
}

But Coverity continued to complain about this line. It still complained that getErrorType() could return a null value and claimed that it was also being dereferenced. Both Miquel and I looked at this and decided that the tool was mistaken. But then today I suddenly realised what was going on. The left hand side of the comparison is a plain old int whereas the right hand side is an Integer. So there is an invisible call to intValue() happening to do the unboxing. Coverity picks this up, but if you just glance at the code you can’t see it. I guess this is the price you pay for auto-boxing - there is code being called that you can’t see…

UPDATE: For some reason I thought the == operator called a method on the left hand operand. It doesn’t, it merely compares the object references. So swapping the order of the operands would make no difference. I think I was fooled by being told that the left hand side was null and being dereferenced. That’s the problem you’d get with a.equals(b) where a could be null. In this case swapping the objects around would make all the difference.

Since Miquel has blogged about his experiences integrating Coverity Prevent with our build process, I thought I would do something similar for Structure101.  I explained what this tool is and how it helps you keep track of dependencies within your codebase in a previous posting.  But that was just my experience of using the standalone client.  Now I’m going to describe how I got it running as part of our continuous integration system.

Unlike Coverity Prevent, Structure101 is quite easy to integrate with ant.  It doesn’t come with ant targets as such, but it does come with a set of java-driven command line utilities.  These are easy to call from ant using the <java> task.  So getting Structure101 to run was fairly easy.

I used the S101Report utility to generate a simple HTML file to show the overview details.  However, it would be nice to show more detail and to be able to see changes over time.  In order to do this you need to have a repository and to publish your results to it.  Again this was not too difficult using S101Publisher. I appreciated the fact that you can label the snapshots you publish, which meant that I could use the build number coming from our continuous integration system and tie the information together.

Once you have published snapshots in the repository, you need to be able to view them.  Like Coverity Prevent there is a web application that acts as a front end to this information.  But unlike Coverity, this webapp can be deployed in a standard servlet container such as Apache Tomcat.  This was good, because we are already running one to provide a web interface to the build system.  So with the web application deployed I could link to the results stored in the repository.

So overall I was pleased with how easy it was to get it working.  The details of the analysis are still slightly difficult to get at, because you have to navigate to the Structure101 web app to see them.  It is possible to get simple graphs out of the web app which could be embedded in a project dashboard or the like.  But there isn’t any documentation for these - you just end up grabbing the URL of the graphs already displayed and tinkering with them.  In an ideal world there’d be some JavaScript that you could embed in a results page that would pull out the required graphs or figures without the need to actually navigate to the web app itself.

I have been introducing Coverity Prevent for Java into our continuous integration stream,  so I decided to post some comments about it.

Installing the software is not difficult at all, just uncompress the files and get a license. Starting to analyze code is also quite easy following the Quick Start Guide, once you get it working you can progressively tune the analysis for your needs . The program supports a great variety of platforms and operative systems so it is very likely that it will work in any kind of machine you happen to have your code and binaries on. It also does a great job detecting failures, quite clever stuff.

However when it comes to integrate Coverity Prevent with your build system it is not as straight forward. You code in Java, don’t you? so it is very likely that you use ant to build the system, but there is no support for ant to run Coverity Prevent. The program runs as a number of native applications or shell scripts called with several parameters, therefore I had to write a few shell scripts to run the different programs with the correct parameters and then use the exec ant task to call those scripts. I would have been quite happy if I could just drop some libs into ant directory and run an hypothetical prevent_analyze task from the ant build script.

Coverity Prevent stores found defects into a proprietary database and you can manage and query this database via web using an embedded web server or via command line. The embedded web server is another annoying thing, as we are already running an application server in the machine for the continuous integration testing, I think it would be good that the web interface was also provided as a java web application deployable into the web application server which you are likely to be using and managing.

Apart from that, the access via web looks nice,  you can see the defect description and explanation. It allows you to classify the error as intentional or a real bug. You can also assign a severity to a given defect, and specify an action to mark the defects to be resolved, expected to be fixed or ignored. When finding defects, it allows you to visualize defects by product, by component into a given product, sort them by any field, etc.

When accessing via the command line you can perform all the same operations and even more, allowing a great degree of automation and produce reports on the information stored in the defect database. This functionality allowed us to extract some statistics on the number of defects and defect listings to be presented in our build results main page. Using the web access was less useful as user need to log into the web application to be able to see results and we wanted some output to embed into the reporting system and have the current state at a glance.

The need to log in and the fact that the graphs produced in the web view are done with interactive flash prevented us to have a nice main page showing graphs along with the statistics. It would have been nice to have the command line query to produce PNG images with those graphs instead of interactive flash applications. It could be done with the query command line and some plotting package such as ploticus or gnuplot but it would require too much time to be invested on.

In conclusion, I have to say that it is a good analysis tool which detects many pitfalls in the code and it really helps, but it is yet another tool to add to the defect management pool of tools as we are already using Jira for that purpose, asking a plugin to integrate them would probably be too much ;)

I recently upgraded my OS X Java development environment to
the Java SE 6 Developer Preview 9 release. After the upgrade I encountered the following error when running code which relied upon the AWT libraries:

seventh-circle$ java AWTExample
2008-04-24 12:15:12.061 java[1035:10b] Apple AWT Startup Exception : *** -[NSCFArray insertObject:atIndex:]: attempt to insert nil
2008-04-24 12:15:12.087 java[1035:10b] Apple AWT Restarting Native Event Thread


Ugoogalizing the stack trace indicated this was a rare but not unknown problem, with no obvious solution. A known unknown, if you will.

After blaming everyone else I retraced my steps and realised that I had changed the CurrentJDK and Current symlinks in /System/Library/Frameworks/JavaVM.framework/Versions to point at 1.6, in a misguided attempt to make this the default version of Java for the shell. Specifically by repointing the Current symlink from A to 1.6 I had broken the Java Native Foundation framework. I reset the symlink and chalked the lost hours up to the arrogance of youth:

cd /System/Library/Frameworks/JavaVM.framework/Versions
sudo rm Current
sudo ln -s A Current

In summary, if you want to change the default version of Java to 1.6 for a command line process on OS X, I’d recommend using the Java Preferences Application. If you want to be sure add the following to your .bash_profile:

JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
export JAVA_HOME
PATH=$JAVA_HOME/bin:$PATH
export PATH


Having been brought up on C# and .Net, Java enumerations are a breath of fresh air - you can do all sorts of fun stuff like give them methods and properties like any other object. Well, almost.

I’ve been writing some code to interact with a payments system. The rules as to whether specific types of payments can be taken are rather procedural, and so don’t instantly lend them to OO’s usual easy encapsulation. Typically I’d end up writing something like this:

public enum ProductType { SHOES, SOCKS, SHIRTS; }
 
public static boolean isBuyable(ProductType product, Account payeeAccount)
{
    // nasty procedural rules and switch statements to 
    // determine whether things are buyable for the given account...
    switch (product)
    {
        case SHOES:  /* ... */ break;
        case SOCKS:  /* ... */ break;
        case SHIRTS: /* ... */ break;
        default:
            // panic! someone added an enum instance 
            // without updating the business process!
            throw new PanicException("run!");
    }
}

This time I thought I’d try putting the rules about product types on the product type itself. Easy enough in Java: stick an abstract method on the enum instances and let them do their own work:

public enum ProductType
{
    SOCKS
    {
        @Override
        public boolean isBuyable(Account payeeAccount)
        {
            // rules about sock availability only
        }
    },
 
    // other product types go here, each looking after its own rules
    // and only its own rules:
 
    SHOES  { /* ... */ },
    SHIRTS { /* ... */ };
 
    // abstract base method for all enum instances in the group
    public abstract boolean isBuyable(Account payeeAccount);
}
 
// the nasty method previously required could then be reimplemented easily:
public static boolean isBuyable(ProductType product, Account payeeAccount)
{
    return product.isBuyable(payeeAccount);
}

The advantages of this approach include:

• Business rules are ‘close to’ the objects they concern. In this case I could have written a method like isBuyable(ProductType) on the account object, but the payments logic is naturally separate from the rest of the system, and this approach keeps it that way.
• Business rules for the availability of specific products are cleanly separated - each product only has to manage its own rules.
• There are no switch statements (the equivalent for enums of the dreaded instanceof for objects) and the associated PanicExceptions.
• It’s just not possible to add enumerated instances that don’t respect the business rules - that abstract prevents their compilation…

Unfortunately, having instance methods on enums breaks their serialisability. Or at least it does with our combination of Hessian and Java for remoting - I’ve not tracked down the precise cause. My first thought was that enum deserialisation was using reflective instantiation under the hood, so I removed the abstract keyword from the root declaration of isBuyable, but no such luck.

A quick refactor put each enumerated instance’s business logic into a discrete interface implementation, but this slightly bloats an otherwise neat solution to the problem. Reality trumps design once again…

IntelliJ, the extremely good Java IDE from Jet Brains will often fail to pick up external file changes; for example performing a subversion “svn up” on your source code.

The problem can be fixed by removing or renaming fslogger in IntelliJ’s bin directory (e.g. /Applications/IntelliJ IDEA 7.0.3.app/bin) as is well documented.

Jet Brains continue to ship this library — enabled by default — for version after version of IntelliJ. Each time I install a new version, the first thing I have to do is disable fslogger and if I forget, then I tend to end up wrongly accusing a colleague of checking-in broken code.

The fslogger library doesn’t work for anyone here as far as I am aware. Shipping a library that actually stops a product from working, when it works so well without it is something I find hard to understand. Perhaps Jet Brains would consider removing this library from the IntelliJ distribution or shipping it disabled by default.

In the recent Jolt Awards you may (or may not) have noticed an award to Structure101 by Headway Software. We’ve been using this tool for a while now and I thought it might be worth giving my impressions. Put simply it allows you to visualise the dependencies within your Java codebase and highlights areas of complexity. It does this using a concept called ‘Excess Complexity’ which Headway Software explain on their website. It shows this complexity at every granularity from top-level package right down to method.

Obviously decisions about this sort of thing can never be entirely automated, but this tool does give you a good way to track down design ‘smells‘ in the code. It may be that the complexity is not a problem, but it certainly gives you cause to reconsider your design decisions in that area.

You may be thinking at this point “But my IDE already shows dependencies, so why would I need to pay good money for this?”. That may be true and I know for a fact that IntelliJ IDEA at least will display a grid of package dependencies for you. But if you decide that you want to unpick some circular dependencies it is very hard to see what’s causing the problem. Compare:

I know which of these is more intuitive. The Structure101 diagram also has the advantage that clicking on the links show exactly which piece of code is causing the dependency.

At the moment we are using the tool to keep our dependencies under control by just analyzing them and graphing the overall complexity figure on a whiteboard. This helps to motivate you to keep it down. We will be fully integrating it with our continuous integration server when we move over to a new system in the next week or so. After that we may consider using the IDE plugins to highlight design problems to individual developers as they work.

Overall, a nice piece of work to keep control of complexity and one that is now free for open source projects.

We have a Cisco CallManager (CCM) VoIP telephone system here at Nominet and I’m currently investigating how this system might be adapted to support ENUM lookups for outbound calls so that calls can be made directly to other VoIP systems without going over the PSTN. There’s no built-in support for ENUM in CCM at all, so some sort of plugin would be required.

I’ve not been able to find any internal plugin APIs, but CCM does support most of Version 1.2 of the Java Telephony API. My hope is that this API will provide tight enough integration with the system to allow the necessary outbound call routeing.

I found that there are very few examples around of how to use JTAPI to actually do anything. Cisco does provide a sample application, but it’s unnecessarily complicated. The code below is therefore presented as a much simplified version of that application that does nothing except tell a CCM extension to dial a specified number.

To use this code you’ll need to obtain the jtapi.jar file from the CCM Administration software via Applications->Plugins->JTAPI. You’ll also need to create a new “Application User”, and add that user into the JTAPI Groups, and also grant that user access to some terminals.

With jtapi.jar in your CLASSPATH, this application can be run as:

% java MakeCall {host} {user} {password} {extension} {number}

As soon as the program has told CCM to place the call it will exit.

As this is only an example there’s only minimal input and error checking, but the application will generate an appropriate error message if you supply incorrect JTAPI login details, or if you try to trigger a call from an extension that your JTAPI user doesn’t have access rights to.

There are two specific parts of the code that deserve special mention:

1. The com.cisco.cti.util.Condition class is not part of the JTAPI standard, but provides a useful semaphore object. In this case it’s used to make the main thread block until the ProvInServiceEv event is received by the anonymous ProviderObserver object.
2. It appears that it’s always necessary to add a CallObserver to the Address object (even if that observer subsequently does nothing useful itself) since the JTAPI library throws an exception if you don’t. I couldn’t find this behaviour documented anywhere.

MakeCall.java

import javax.telephony.*;
import javax.telephony.events.*;
import com.cisco.cti.util.Condition;
 
public class MakeCall
{
	public MakeCall(String[] args) throws Exception
	{
		String hostname = args[0];
		String login = args[1];
		String passwd = args[2];
		String src = args[3];
		String dst = args[4];
 
 		/* start up JTAPI */
		JtapiPeer peer = JtapiPeerFactory.getJtapiPeer(null);
 
 		/* connect to the provider */
		String providerString = hostname;
		providerString += ";login=" + login;
		providerString += ";passwd=" + passwd;
		Provider provider = peer.getProvider(providerString);
 
		/* wait for it to come into service */
		final Condition	inService = new Condition();
		provider.addObserver(new ProviderObserver() {
			public void providerChangedEvent (ProvEv [] eventList) {
				if (eventList == null) return;
				for (int i = 0; i < eventList.length; ++i) {
					if (eventList[i] instanceof ProvInServiceEv) {
						inService.set();
					}
				}
			}
		});
		inService.waitTrue();
 
		/* get an object for the calling terminal */
		Address srcAddr = provider.getAddress(src);
		srcAddr.addCallObserver(new CallObserver() {
			public void callChangedEvent (CallEv [] eventList) {
				/* ignored */
			}
		});
 
		/* and make the call */
		Call call = provider.createCall();
		call.connect(srcAddr.getTerminals()[0], srcAddr, dst);
	}
 
	public static void main(String[] args) {
		try {
			new MakeCall(args);
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			System.exit(0);
		}
	}
}

Here at Nominet we use Hessian to build our Java middle-tier services. We’ve been using version 3.0.20 for a while and, once you get used to its idiosyncrasies, it’s pretty workable and stable.

Yesterday I tried to update to the latest version, 3.1.3. Suddenly all hell broke loose. A bunch of domain entity objects we’ve been persisting through Hibernate stopped deserialising cleanly at the Java client end. A little debugging revealed that:

• Instances of java.sql.Timestamp would drift by a few seconds from their transmitted version. We’d been using just such fields along with a @Version annotation to provide optimistic locking for Hibernate-managed entities. Lots of StaleObjectStateExceptions get thrown.

“No problem” I thought, “they’re just there because we haven’t moved them over to cleaner, less coupled Integer versioning yet. Refactor!”. Half an hour of refactoring class and table definitions later, and the local unit tests pass again. Back to service-level tests:

• Deserialisation now fails for a couple of alternating reasons, each of which throws a HessianFieldException:
  • Enumerations randomly fail to deserialise, complaining that they couldn’t be assigned from HashSets;
  • Sets of enumerations also fail, with a root IndexOutOfBoundsException from an ArrayList.

At this point I gave up and reverted to the previous version of the library. Another developer here also had the same problems with the updated version, so I think that for now we’ll be sticking with the tried and trusted version 3.0.20. Just because a new library is there doesn’t mean you have to upgrade straight away, or even at all.

Having said that, if anyone can shed any light on these problems it would be very much appreciated…

This probably isn’t news to many but I only recently noticed that Oracle AQ can manage JMS queues. We don’t use a full blown J2EE application server so our options when using a JMS broker is to either have a standalone server/cluster (more work for SAs) or to embed the JMS server in to our middleware (more work for me). By using the database we add no new dependencies or software so this is a pretty neat solution. Another big advantage for us is that it looks like we would be able to create messages inside PL/SQL procedures but I may be getting ahead of myself here.

There isn’t much to it but it was a lot harder to get to the same point than with the other JMS brokers I have been testing. There is a lot of documentation to go through and it is all fairly general. I also couldn’t find a good example so I’ve created a quick howto… Continue Reading »