The Frustrated Architect:

Software architecture plays a pivotal role in the delivery of successful software yet it’s frustratingly neglected by many teams. Whether performed by one person or shared amongst the team, the architecture role exists on even the most agile of teams yet the balance of up front and evolutionary thinking often reflects aspiration rather than reality.

(Via coding the architecture.)

Update – if you go to this page you can get a video of the presentation; it’s about an hour long. A word of warning: I couldn’t make the video play on the site with Chrome, I had to use Safari.

Java is not the new COBOL | Craig Tataryn’s .plan:

A switch statement that I’ll actually use
Yes, finally my Java brethren we have a switch statement that actually works on Strings!

switch (lang) {
   case "Java" :
      out.println("I like frameworks!");
      break;
   case "Ruby" :
      out.println("I like Pabst Blue Ribbon!");
      break;
   case "PHP" :
      out.println("I like WordPress!");
      break;
}

My God this has been a long time coming.

The technology world has just lost another giant, though one without the towering public persona of Steve Jobs.

If you’re not actually a programmer, you’ve probably never heard of Dennis Ritchie. But the vast majority of software you use was built using a tool that he originally designed, and the rest by tools that very liberally sample from his.

The “native language” that the central processor in a computer understands is an ornery beast. For one thing, back in the 1970s every two-bit computer company (if you’ll pardon the techy pun) had its own native language; these days, there still remain two very common ones, and dozens of less common examples out there. More importantly, it’s almost incomprehensible, even to most programmers. Take this little snippet, part of the preliminaries to a very simple program that just prints the message “hello, world” to the screen:

_start:
        mov    eax, 4
        mov    ebx, 1
        mov    ecx, str
        mov    edx, str_len
        int    80h

Writing long and complicated bits of software with such unhelpful notation is extremely slow and error-prone.

“High-level” languages, that allowed the logic of software to be expressed in more compact and readable notation, had existed since the 1950s; Grace Hopper was responsible for one of the first. Over time, more and more of the scientific and business software run on the large computers of the era was written in FORTRAN, COBOL, and other high-level langages. However, the “operating systems”, the software plumbing that joined those applications to the hardware, was invariably written in the machine language of specific systems.

In the late 1960s, Ritchie, working with Ken Thompson at Bell Laboratories, hacked together their own little operating system for an obsolete computer nobody was making use of. It was small, but it worked, and was one of the first practical systems to support “timesharing” – the ability for multiple users to run multiple programs simultaneously and interactively. Fairly early on, they had another brainwave; they would rewrite as much of the system – which became known as Unix – as possible in a high-level programming language, to speed development. But first, they needed a suitable high-level language. The resulting language, an evolution of earlier languages entitled BCPL and B, was called “C”.

Both C and Unix were raging successes, partly because of their inherent strengths. The use of C allowed Unix to be “ported” to many different computer systems, a process that continues today as its spiritual successor Linux, written in C, runs on everything from IBM mainframes (and the amphormous “Googleplex” of Google’s servers which, reportedly, draw 240 megawatts of power), to virtually every smartphone on the planet (the iPhone’s operating system is not Linux, but it is also a derivation of Unix and substantial parts are written in C). They also had the good fortune, as with the Internet and World Wide Web which owes so much to both, that they gradually escaped the crush of intellectual property law to become part of the intellectual commons of the field.

Almost as important was the elegance and economy that Ritchie, along with Brian Kernighan, brought to teaching the language. Their textbook The C Programming Language remains the best programming language textbook ever written, in my view, and the one that I still strongly recommend to my students.

Much of the Windows operating system, and Mac OS X, are implemented in C. Those parts that aren’t, are implemented in computer languages directly derived from it – C++ and Objective-C. Most of the software that runs on those systems is also written in C or its successor languages. And perhaps the most pervasive “new” high-level language of the last 20 years – Java – retains so much of C’s “look and feel” that it often takes a second glance to tell which language a piece of code is written in.

Neither C nor Unix were by any means perfect. While some of its design faults have been eliminated in its successors others remain, and will likely continue to bamboozle neophyte (and, all too often, experienced) programmers, for generations to come. But there was so much he and his colleagues got right. Fate did play its part, but there are very good reasons that generations of software developers not yet born will express themselves in notations largely based on Ritchie’s.

RIP, dmr.

via Dennis Ritchie: The Shoulders Steve Jobs Stood On | Wired Enterprise | Wired.com.

It all started when I needed to start the Transaction out in the view, i.e. as soon as the resource is opened on the HTTP side (rather than when the database service layer is called). I’m using JPA for a number of reasons;

1. I need to access multiple databases each with a different schema (this means different connections)
2. They need to have XA transactions.
3. I’d like the transactions managed by the container (JTA).

JPA provides all these things easily without all the complex Hibernate.xbm.xml mapping files and what-have-you.

The trick to starting the transaction with the web session is to use spring’s OpenEntityManagerInViewFilter. Unfortunately I was using  PersistenceAnnotationBeanPostProcessor to manage my multiple persistence units. The filter wants to know what EntityManagerFactory it should bind to, and fair enough. But with my minimal configuration, there was no addressable EntityManagerFactory!

The solution was to stop the PersistenceAnnotationBeanPostProcessor from doing the JNDI look ups and bind each PersistenceUnit into the Spring context with a manual JNDI lookup;

  <tx:annotation-driven />
  <tx:jta-transaction-manager />

  <bean id="pabpp" 
   class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor" />

  <jee:jndi-lookup id="onePU" jndi-name="persistence/onePU" />
  <jee:jndi-lookup id="twoPU" jndi-name="persistence/twoPU" />
  <jee:jndi-lookup id="threePU" jndi-name="persistence/threePU" />

then, in the web.xml the ViewFilter could be bound explicitly to the required persistence unit, in this case “onePU”. A fuller explanation can be found on my spring source forum post here; http://forum.springsource.org/showthread.php?115844-OpenEntityManagerInViewFilter-with-JPA-PersistenceAnnotationBeanPostProcessor

Overall, this is now quite an elegant solution.

UPDATE. More documentation of the solution and the various configurations at the Spring forum here: http://forum.springsource.org/showthread.php?115587-Example-for-using-two-databases-w-Spring-amp-JTA-transaction-manager&p=383432#post383432

Our system has generic components which “pass off” specific knowledge of specific data sets to specific processing components, where there are not a fixed set of these specific components available to all the generic components. In classic OO design this is achieved with interfaces and many design patterns like the Visitor Pattern. In our design we’ve replaced much of the complexity of the interface with REST-based architecture. Our application has become far more modular (despite its run-time complexity). More importantly, we are at the the point where we can easily start delivering the advanced features that the business has now specified for us without introducing a massive bloat-o-burger one-size-fits-all set of “common data definitions” that are the union of all possible sub-system functionalities.  Each sub-system can have its specific features required for that data set and still satisfy the handful of generic actions it is required to support. Each of these specific features can be developed in isolation. It’s a big win for us.

In Eclipse’s defence, Intellij IDEA and Netbeans also exhibit identical broken functionality.

For a start, the documentation is merely a series of outlines of brief hints. One measly section.The laughably short Spring 3 doco section 13.5.1.4 “Dealing with multiple persistence units” conveniently omits  the transaction manager configuration from the example. The problem appears to be that the JPA transaction manager only (and compulsorily) deals with one entity manager factory. And an entity manager factory only deals with one persistence unit. So therefore you have to have two JPA transaction managers. Which means the two persistence unit transactions won’t co-operate properly even though they may share the underlying datasource and are configured through the single persistence unit manager bean.

The above scenario is totally trivial in an EJB 3 container backed with Hibernate, or Eclipselink, as the provider. About one-quarter of the configuration. If I have to use a JTA transaction manager obtained by JNDI lookup from the container, to run a transaction across two JPA persistence units which share the same underlying datasource, why the hell am I using Spring in the first place?

All I wanted was to isolate one set of db transactions from another, use JPA persistence units so the two sets of tables could live easily in two different schemas, and use the annotations to kill the fragile AOP regex-like-but-not-like-regex class and method pattern-matching jiggery pokery from the Spring configuration.

My god, this is such a stonkingly non-trivial forest of horrible configuration and a trial and error morass of filthy swamp miasma … an equivalent EJB3 backed by Hibernate JPA set up is, by comparison, an almost effortless task. I can produce a unit-tested all singing all dancing multiple persistence unit JPA app running in an EJB 3 container with XA datasources using Hibernate as the JPA implementation in almost no time at all. Getting the same thing with Spring (oh, and running some tests with Jetty inside Maven) is like having all your teeth pulled while you’re coming down from a three day methamphetamine binge. Getting Spring to run with multiple JPA persistence units on the same JDBC connection (without XA, across the same database connection) with a transaction propagation of REQUIRES_NEW on the entry point of one of the units is a hair-pulling, beard-greying, head-desk banging, co-worker punching, drunken ranting, blood-pressure raising experience of pure horror. Which apparently doesn’t end.

Die, Spring, die. Can’t come soon enough as far as I am concerned. It’s more evil than Oracle. At least you know with Oracle you’re in for an un-lubed hard and fast backdoor job from Ellison with no reach-around before you even unpack the box. Spring is like a beautiful young sexy soft-porn film that turns into some relentlessly horrific succubus-filled horror film half way through.

Seriously, its enough to make one pine for bloody Weblogic. Spring is the new EJB 2.1.

Recently I was learning ANTLR, trying to decide whether this was a right technology to pursue a particular project which involved parsing a preexisting message format. After a week of a spike, we decided that it was worth pursuing and started on earnest on the grammar for our project. However a week into this process, I had an epiphany … I was doing some things wrong with the ANTLR grammar which were now slowing progress in adding the new characteristics it needed to be complete. Many developers know this feeling; the features of my grammar that I had built over the first week were naive and now hampering it from expanding into the new requirements. I took it on myself to kill the entire grammar and start again. It took less than a day and half to replicate that week’s worth of work (i.e. pass the test suite which had built up around it).  I’ve done this before; scrap the first attempt at building a domain and try again. Here my domain was the same (it was after all defined in both a standards specification and in the many hundreds of thousands of sample messages we captured from an existing system), but its implementation needed refinement.

So I think that the rule about throwing away spikes can in fact be made a general axiom of programming:

When you are learning a new technology, make sure you throw away the first thing you build that works – to avoid accumulating your mistakes.

Thanks to Robert for the important qualifier “that works”.

N.B. my views about system rewrites have not changed regardless.

There is a web service inside a component, let’s call that component a ‘Node’, that receives something like (but not identical to!) an Event on its interface. Inside the Event is some data that the Node does not particularly care about (and actually has no access to – it’s just a byte[] as far as the Node can tell). However, the Node contains a Registry which enables components, lets call them Event Handlers, to register themselves with the Node, as available to process certain Events (i.e. decode that byte[] and do something with it) according to criteria which the EventHandler injects into the Node’s Registry.  EventHandler is an interface with a handful of simple methods related to handling the Event, and also registration with the Registry.

So, the process flow looks something like this: The Node first records the reception of the Event at the interface in a log. Then it tells the Registry about the Event, and the Registry produces the EventHandler(s) it needs to use.  The Registry gives the Node back the instances of the EventHandler interface. The Node then hands off the Event to the EventHandler, which does whatever it does unbeknownst to the Node, and returns a fairly simple EventResponse object. The Node records the EventResponse object in its log (i.e. a database) and returns it as the response to the web service call.

Consider the Node service method as looking something like this (you’ll have to excuse the Java 1.4-ness of this code, the WordPress code highlighting plugin apparently hates Java 5) :

  public List receive(Event event) {
    List responses = new ArrayList();
    for (EventHandler handler : this.registry.lookup(event.getMetadata())) {
      Response response = handler.handle(event);
      saveResponse(response, event, handler);
      responses.add(response);
    }
    return responses;
  }

To enable the wiring, the Registry has a method:

  void register(EventMetadata metadata, EventHandler handler)

and among other methods the EventHandler interface defines:

  void setRegistry(Registry registry)

Currently the WAR file imports the Node.JAR and a group of EventHandler.JAR files which are specific implementations for handling different kinds of Events. We configure it in Spring currently so that the specific EventHandler is injected with the Registry object (from the Node.JAR). The EventHandler implementation then registers itself with the Registry in a call-back operation, telling it what sort of Events it will handle.

This all works just fine at the moment. The problem with what we have is that it is all currently statically compiled into the WAR file.The WAR file specifies a Spring application context XML file which in turn loads the Spring configuration for the Node and Registry component, and every Spring application context for each Handler JAR included inside the WAR file’s WEB-INF/lib directory.

Now, we now don’t want every deployed instance of every Node to handle every possible Event. Currently we’ve got a small .properties file that actually tells another Spring component which EventHandlers are be to be instantiated or not. This is working fine when we only desire some Nodes to handle maybe one or two of a larger group of related Events. That is, where we currently don’t mind that the WAR file is identical in every respect on every Node — it’s just that each node contains a .properties file in its classpath that tells it which EventHandlers it is allowed to load and use (and therefore what Events it is capable of receiving, bearing in mind that when I say ‘Events’ there is really only one concrete type of Event, I mean the encrypted data which is held within the Event which is actually consumed by the Handler).

However, we are now in a situation where we want to use this same architecture for a completely different group of Events. We definitely don’t want to have to compile and assemble a new WAR file for different Nodes based on the Event cluster. We want to deploy a standardized Node.WAR which has available on its classpath a dynamic set of XxxEventHandler.JAR which can dynamically register themselves with the Node’s registry. There may be also a requirement for some related custom extension points in the future.

Initially we considered OSGi as the technology to enable this. After some discussion yesterday with people who know better about OSGi, this approach was rejected as impractical for the moment. Therefore last night I set out tooling about with the Spring 2.5.6 ApplicationContext and its related objects to see what could be done to enable dynamically-loaded JAR files within a parent Spring application context. Here is what I’ve discovered we can do, with only some small restrictions on developers writing the individual EventHandler implementations.

The first issue is, we need to locate a set of Spring application context XML file which should be on the classpath but not yet instantiated into any live Spring context.

Assuming we’re in a bean that’s ContextAware or otherwise has access to the Spring application context which is loading it, after the parent context has loaded and initialized (there are method hooks for this sort of thing) we can use the org.springframework.core.io.support.PathMatchingResourcePatternResolver class to search the classpath for a resource:

  PathMatchingResourcePatternResolver pmrl = new PathMatchingResourcePatternResolver(context.getClassLoader());
  Resource[] resources = pmrl.getResources(
    "classpath*:/net/crazymcphee/dynamiccontext/*/Crazy*DynamicContext.xml"
  );

Looking at the Spring 2.5.6 source code we found that the String passed to the getResources() method isn’t a proper regular expression, which is a pity. So you can’t do something like look for **/Crazy*DynamicContext.xml and expect to match a file Crazy*DynamicContext.xml in any package. Also we found that it had to prefixed with that classpath*: … yes, the asterix, literally … else it wouldn’t search the classpath, as opposed to the file path. So we’re restricted in the above example to a file called Crazy<something>DynamicContext.xml in a package exactly one deep from net.crazymcphee.dynamiccontext … e.g. net.crazymcphee.dynamiccontext.package.CrazyMcpheeDynamicPackage.xml matches but net.crazymcphee.dynamiccontext.package.sub.CrazySubDynamicPackage.xml and net.crazymcphee.dynamiccontext.CrazySuperDynamicPackage.xml do not. Therefore we will have a restriction on what package the dynamic context can be in and what it’s name will be. I don’t think that’s too onerous on our developers – we just have to pick a sensible standard.

The next part of the problem is that we have to load the ‘Resource’ thus found into a Spring context. This is pretty easy:

 for (Resource r : resources) {
   GenericApplicationContext createdContext = new GenericApplicationContext(context);
   XmlBeanDefinitionReader reader = new XmlBeanDefinitionReader(createdContext);
   int i = reader.loadBeanDefinitions(r);
 }

The int i will be set to the number of beans found in the createdContext. The createdContext will have the original context as its parent context, so it can gain access to any beans defined there (and also, although we are yet to test this (!), it should also be intercepted by the AOP-based transaction interceptors in the parent, and so forth).

The only other part of the puzzle may be to query the createdContext to see if it has any target beans within it, luckily for us an ApplicationContext has a method getBeansOfType(Class clazz) which will load all the beans of a particular type:

  for (Resource r : resources) {
    GenericApplicationContext createdContext = new GenericApplicationContext(context);
    XmlBeanDefinitionReader reader = new XmlBeanDefinitionReader(createdContext);
    int i = reader.loadBeanDefinitions(r);
    Map map = createdContext.getBeansOfType(EventHandler.class);
    for (Object o : map.keySet()) {
      EventHandler handler = (EventHandler) createdContext.getBean((String) o);
      // do some programmatic manipulation with the EventHandler that we found here
    }
  }

Using these three Spring features will enable us to be able to place a JAR file containing an interface implementation, and a Spring context XML file matching a particular pattern, into the classpath of our WAR, and on restart, we can dynamically pick up the newly inserted features into our application installation. I’ll report back when we get an actual production prototype together that can do this. Hopefully it will be OK to put the code in the blog too (if we make it generic enough).

If you have any further ideas or refinements to this idea, please leave them in the comments!