There is a web service inside a component, let’s call that component a ‘Node’, that receives something like (but not identical to!) an Event on its interface. Inside the Event is some data that the Node does not particularly care about (and actually has no access to – it’s just a byte[] as far as the Node can tell). However, the Node contains a Registry which enables components, lets call them Event Handlers, to register themselves with the Node, as available to process certain Events (i.e. decode that byte[] and do something with it) according to criteria which the EventHandler injects into the Node’s Registry.  EventHandler is an interface with a handful of simple methods related to handling the Event, and also registration with the Registry.

So, the process flow looks something like this: The Node first records the reception of the Event at the interface in a log. Then it tells the Registry about the Event, and the Registry produces the EventHandler(s) it needs to use.  The Registry gives the Node back the instances of the EventHandler interface. The Node then hands off the Event to the EventHandler, which does whatever it does unbeknownst to the Node, and returns a fairly simple EventResponse object. The Node records the EventResponse object in its log (i.e. a database) and returns it as the response to the web service call.

Consider the Node service method as looking something like this (you’ll have to excuse the Java 1.4-ness of this code, the WordPress code highlighting plugin apparently hates Java 5) :

  public List receive(Event event) {
    List responses = new ArrayList();
    for (EventHandler handler : this.registry.lookup(event.getMetadata())) {
      Response response = handler.handle(event);
      saveResponse(response, event, handler);
      responses.add(response);
    }
    return responses;
  }

To enable the wiring, the Registry has a method:

  void register(EventMetadata metadata, EventHandler handler)

and among other methods the EventHandler interface defines:

  void setRegistry(Registry registry)

Currently the WAR file imports the Node.JAR and a group of EventHandler.JAR files which are specific implementations for handling different kinds of Events. We configure it in Spring currently so that the specific EventHandler is injected with the Registry object (from the Node.JAR). The EventHandler implementation then registers itself with the Registry in a call-back operation, telling it what sort of Events it will handle.

This all works just fine at the moment. The problem with what we have is that it is all currently statically compiled into the WAR file.The WAR file specifies a Spring application context XML file which in turn loads the Spring configuration for the Node and Registry component, and every Spring application context for each Handler JAR included inside the WAR file’s WEB-INF/lib directory.

Now, we now don’t want every deployed instance of every Node to handle every possible Event. Currently we’ve got a small .properties file that actually tells another Spring component which EventHandlers are be to be instantiated or not. This is working fine when we only desire some Nodes to handle maybe one or two of a larger group of related Events. That is, where we currently don’t mind that the WAR file is identical in every respect on every Node — it’s just that each node contains a .properties file in its classpath that tells it which EventHandlers it is allowed to load and use (and therefore what Events it is capable of receiving, bearing in mind that when I say ‘Events’ there is really only one concrete type of Event, I mean the encrypted data which is held within the Event which is actually consumed by the Handler).

However, we are now in a situation where we want to use this same architecture for a completely different group of Events. We definitely don’t want to have to compile and assemble a new WAR file for different Nodes based on the Event cluster. We want to deploy a standardized Node.WAR which has available on its classpath a dynamic set of XxxEventHandler.JAR which can dynamically register themselves with the Node’s registry. There may be also a requirement for some related custom extension points in the future.

Initially we considered OSGi as the technology to enable this. After some discussion yesterday with people who know better about OSGi, this approach was rejected as impractical for the moment. Therefore last night I set out tooling about with the Spring 2.5.6 ApplicationContext and its related objects to see what could be done to enable dynamically-loaded JAR files within a parent Spring application context. Here is what I’ve discovered we can do, with only some small restrictions on developers writing the individual EventHandler implementations.

The first issue is, we need to locate a set of Spring application context XML file which should be on the classpath but not yet instantiated into any live Spring context.

Assuming we’re in a bean that’s ContextAware or otherwise has access to the Spring application context which is loading it, after the parent context has loaded and initialized (there are method hooks for this sort of thing) we can use the org.springframework.core.io.support.PathMatchingResourcePatternResolver class to search the classpath for a resource:

  PathMatchingResourcePatternResolver pmrl = new PathMatchingResourcePatternResolver(context.getClassLoader());
  Resource[] resources = pmrl.getResources(
    "classpath*:/net/crazymcphee/dynamiccontext/*/Crazy*DynamicContext.xml"
  );

Looking at the Spring 2.5.6 source code we found that the String passed to the getResources() method isn’t a proper regular expression, which is a pity. So you can’t do something like look for **/Crazy*DynamicContext.xml and expect to match a file Crazy*DynamicContext.xml in any package. Also we found that it had to prefixed with that classpath*: … yes, the asterix, literally … else it wouldn’t search the classpath, as opposed to the file path. So we’re restricted in the above example to a file called Crazy<something>DynamicContext.xml in a package exactly one deep from net.crazymcphee.dynamiccontext … e.g. net.crazymcphee.dynamiccontext.package.CrazyMcpheeDynamicPackage.xml matches but net.crazymcphee.dynamiccontext.package.sub.CrazySubDynamicPackage.xml and net.crazymcphee.dynamiccontext.CrazySuperDynamicPackage.xml do not. Therefore we will have a restriction on what package the dynamic context can be in and what it’s name will be. I don’t think that’s too onerous on our developers – we just have to pick a sensible standard.

The next part of the problem is that we have to load the ‘Resource’ thus found into a Spring context. This is pretty easy:

 for (Resource r : resources) {
   GenericApplicationContext createdContext = new GenericApplicationContext(context);
   XmlBeanDefinitionReader reader = new XmlBeanDefinitionReader(createdContext);
   int i = reader.loadBeanDefinitions(r);
 }

The int i will be set to the number of beans found in the createdContext. The createdContext will have the original context as its parent context, so it can gain access to any beans defined there (and also, although we are yet to test this (!), it should also be intercepted by the AOP-based transaction interceptors in the parent, and so forth).

The only other part of the puzzle may be to query the createdContext to see if it has any target beans within it, luckily for us an ApplicationContext has a method getBeansOfType(Class clazz) which will load all the beans of a particular type:

  for (Resource r : resources) {
    GenericApplicationContext createdContext = new GenericApplicationContext(context);
    XmlBeanDefinitionReader reader = new XmlBeanDefinitionReader(createdContext);
    int i = reader.loadBeanDefinitions(r);
    Map map = createdContext.getBeansOfType(EventHandler.class);
    for (Object o : map.keySet()) {
      EventHandler handler = (EventHandler) createdContext.getBean((String) o);
      // do some programmatic manipulation with the EventHandler that we found here
    }
  }

Using these three Spring features will enable us to be able to place a JAR file containing an interface implementation, and a Spring context XML file matching a particular pattern, into the classpath of our WAR, and on restart, we can dynamically pick up the newly inserted features into our application installation. I’ll report back when we get an actual production prototype together that can do this. Hopefully it will be OK to put the code in the blog too (if we make it generic enough).

If you have any further ideas or refinements to this idea, please leave them in the comments!

But the bigger headache was the Java libraries for handling WebDAV. There is the now-abandoned Apache Slide, the based-on-Slide webdavclient4j, and the way-more-complex-than-it-needs-to-be Jackrabbit. Slide and webdavclient4J have the appearance and feel of crusty, molten code from 2004. Jackrabbit is a full-featured server product … to quote, it is a “hierarchical content store with support for structured and unstructured content, full text search, versioning, transactions, observation” … and I’m sure that’s real nice if that’s what you want, but I just need to automate push a file onto an Apache HTTPD instance and tell another service what URL it now lives at. It’s not rocket science. Jackrabbit has too many features, and too little simple documentation, to be truly useful for what I need.

Enter the easy to use WebDAV client, Sardine. Here is the author’s post announcing it.  It’s very simple to use:

Sardine sardine = SardineFactory.begin();
List<DavResource> resources = sardine.getResources("http://yourdavserver.com/adirectory/");
for (DavResource res : resources)
{
     System.out.println(res);
}

Anyway, simplicity … just the sort of thing I was looking for. Recommended.

The problem for Glassfish, as the second sentence of this ServerSide article states (see it straight from Oracle’s mouth here, and see also here) is that Oracle view it as being used for “non-mission critical department apps”. Glassfish’s superior technology (or otherwise) just doesn’t come into it. It’s not a factor (as it rarely every is).

Not so long ago Oracle spent a big wad of money acquiring an app server (Weblogic) and then a stack of more money porting all its other products into it and branding the resulting mess platform “Oracle Fusion Middleware 11g”. Now not only do they have their third app server (OC4J/OAS, Weblogic and Glassfish), but the Sun product suite includes products that compete with various Fusion Middleware 11g products (portals, ESBs, and so on). So on one hand you’ve got a “departmental” application server, which you can either licence for free by downloading the open-source version, or buy support for the fancier ‘Enterprise’ version, and on the other, an expensive, full-stack-integrated (all the way to the IDE), fully-branded strategic platform that Oracle just invested a vast amount of money into, and have been pushing like crazy onto customers the past six months. And it is the same sales team that will sell both this licensed “departmental” Glassfish. Therefore if you say the magic words like “need a cluster” or maybe “we might build a portal”, or “we are considering adopting a service-orientated architecture”, lo and behold you’ll find the molto-dinero “Fusion Middleware” based solution installed all over your sorry arse quicker than you can say “can you please explain this per-core with special CPU-architecture-loading-factor licencing schema to me once again and why is it a different price if I upgrade my hardware without adding any additional cores???“.

Let’s dissect those “key points” of Oracle’s strategy announcement:

Key Point	What they meant to say
GlassFish continues as the Java EE reference implementation and as an open source project.	We see it as the way to dominate the direction of Java EE for at least two years, but for Larry’s sake don’t try to use it in production.
Oracle’s strategic application server, Oracle WebLogic Server, together with GlassFish, provide world class Java EE infrastructure.	Oracle’s strategic application server, Oracle WebLogic Server something something something provide world-class something something infrastructure.
GlassFish Enterprise Server and WebLogic Server expected to share core components.	We are the Borg. Resistance is futile. You will be assimilated.
Oracle plans to add GlassFish Enterprise Server all WebLogic offerings.	Hey, look at this cute free “reference implementation” thingy that comes free with Weblogic! You could use that to run your departmental Wiki instead of having to pay us another fortune for more Weblogic licences. Did you say “WIKI”? Did we tell you all about the great wiki-like Enterprise 2.0 features available in the Oracle Fusion Middleware 11g offering? How many test environments did you say you needed licences for?
GlassFish Web Stack maintained for existing customers.	Not available for sale.
GlassFish Message Queue remains as the GlassFish messaging infrastructure.	We’re not expecting to sell any licences of this. Just use Oracle Fusion Middleware’s SOA Suite 11g already. We’re fairly sure that’s got a message queue in it.
Oracle plans to license GlassFish Enterprise Server and Java System Web Server with all WebLogic Server offerings.	See above.
GlassFish also available as standalone offering.	Are you sure you didn’t mean to say “Weblogic”? No? Can you call back next Thursday at 2pm and ask for Fred? We’re reasonably certain he might know something about that Glassthingy.
GlassFish will continue to be supported and maintained for an extended time period for customers current on support.	Well, the lawyers said we had to. We know how to do this. Ask any 10g customer.
GlassFish open source projects thrive	As long as we will let them.

I know I’m a completely cynical bastard about these things, but I will wager within a few months that even if you deliberately ask for Glassfish Enterprise directly that you’ll have to fight off the Weblogic borg absolutely tooth and nail to the last man as they repeatedly try to board your IT department brandishing their integrated-wizard-driven Red Stack. I predict that, basically, after a year of not even trying to sell any Glassfish licences – because if you ask for any of the features that are in the licenced version and not the open-source one, you’ll be pushed to Weblogic (and anyway, at ten times the price they’ll prefer to sell you Weblogic as a default position, after all “Glassfish comes free with Weblogic”) – Oracle will announce, “there’s no sales in it”, then probably ditch the licenced Glassfish version completely, leaving only the open source version. Finally sometime after that they’ll cut the open source funding off and it will have to limp along without hardly any of the resources it formerly had. Maybe they’ll donate it to the ghetto of an Apache incubator project where it can die unnoticed a couple of years after that. It’s a pity because IMHO Glassfish is ten thousand times a better app server than anything Oracle ever produced, or even bought before this.

Collection mapping is one of those “hello world” problems. (The “hello world” example in the doco looks totally trivial and completely ideal [which is the problem], but suck-in-the-galaxy-greet-it-and-then-map-all-the-stars problem, which is more like what your real app looks like, is far less than trivial and not so easily achieved with the demonstrated toolkit). In my experience, collection mapping is the cause of all sorts of pain and performance perfidy.

Here’s a further situation where I find that they really get in the way. Consider integration with existing systems that use highly-normalised database schemas (designed by advanced SQL programmers who know what they are doing in that language).

This is not an uncommon scenario – we have this system ‘A’ which (let’s say) has all our customer shipping, product, warehousing data in it. We don’t want to replace it. But we need to extend it so that e.g. available to web systems. Or maybe it needs to be integrated into the big fancy off-the-shelf CRM we’re installing. Something like that. We need to build some sort of service layer around this system ‘A’ so that other systems can access its data and/or functions. It’s common to think that an ORM around that big complex database schema is going to help, but in my experience, it doesn’t for many of the reasons Stephan and Aldo list. That sort of highly-normal database schema in my experiences completely kills ORM object representations stone cold. You end up with so many LazyInitialisationExceptions and various other problems it sucks productivity out the developers and performance out of the system.

Also, I get a bit annoyed about ORM abstractions leaking into the web tier, this is especially prevalent with using annotations rather than ORM mapping files.

Different sorts of approaches are sometimes needed to be applied with careful thought in that scenario. Perhaps you need to model the “intermediate” domain you need directly, and then use, for example, simple DAO layers which operate directly on the domain you’ve defined. These DAO layers then might call functions (stored procs) in the database that do the problematic mapping in an efficent relational language (after all it’s the direct representation of the target data format).

I’m not saying that you have to do the above in every case, or that an ORM is always wrong, but anyway the point is, ORM isn’t a magic bullet like any other technology you have to consider carefully. Your application doesn’t always need one by default.

The basics of Tapestry is that it is a component-based web framework. Just about everything, including web pages, are components. Components may contain other components. The way it works is very simple and quite elegant, once you get used to it and weaned off the big-XML-file style of configuring a web application.

When you start, you have two main Java packages that are created for you (if you use the maven archetype, otherwise you will create these packages yourself). If your package root is say “net.crazymcphee.webapp”, then your two packages are “net.crazymcphee.webapp.pages” and “net.crazymcphee.webapp.components”. To configure a Tapestry 5 project with maven use this command and answer the prompts:

mvn archetype:generate \

    -DarchetypeCatalog=http://tapestry.formos.com/maven-snapshot-repository

Don’t use the one in the tutorial as it will not work! This is an excellent illustration of the first and most serious problem that Tapestry 5 has: the documentation not only has massive lacunas, it is also sometimes wrong and not updated.

Now, any class that you create in the “pages” package will automatically become a page in your application. But these classes need not be very complex at all. For example:

package net.crazymcphee.webapp.pages;
 
import java.util.Collections;
import java.util.List;
 
import org.apache.tapestry5.annotations.Persist;
import org.apache.tapestry5.annotations.Property;
import org.apache.tapestry5.ioc.annotations.Inject;
 
import net.crazymcphee.webapp.model.Person;
import net.crazymcphee.webapp.services.PersonService;
 
public class Persons {
 
    @Inject
    private PersonService personService;
    @Property
    @Persist
    private List persons;
    @Property
    @Persist
    private String searchTerm;
 
    Object onSubmitFromSearch() {
        persons = personService.findPersons(searchTerm);
        return this;
    }
 
    Object onSubmitFromClear() {
        persons = Collections.EMPTY_LIST;
        return this;
    }
 
}

You can see that this class doesn’t extend any infrastructure classes and has quite a simple structure. This is a fully functional page with two actions – one populates a list, the other clears it.

Now, to explain just a little here; PersonService is injected by Tapestry, but you need to configure in your AppModule which implementation you want to use. You can also use it with Spring as your IOC container but the one that comes with Tapestry is perfectly good enough for most applications.

The List, persons, and the searchTerm parameters are marked as @Property (so we don’t need to add getters and setters) and also @Persist so that the variables are preserved from request to request.

The methods “onSubmitFromSearch” and “onSubmitFromClear” are using a convention – “onSubmit” will also work, but assuming (as is true in this case) that we may have multiple forms on the one page, each method will only be fired from the “search” form in one instance, and the “clear” form in the other. These names are not special, it’s just (as you’ll see below) the forms will have these two names. They could be “Bill” and “Ben” in which case the methods would be “onSubmitFromBill” and “onSubmitFromBen”.

You will also note that these methods return the same page instance, which tells Tapestry to re-render the same page, but if you wanted to forward onto another page, you would add a instance variable, mark it with an @InjectPage annotation, and return that instance variable (after initialising it in your “onSubmit” method) instead of just returning “this”.

This page Persons, is available at {application-context-path}/persons. But there is one other part of the puzzle – the actual view. As mentioned, Tapestry uses convention over configuration and in this case, the convention is that the Page markup must be named the same: Persons.tml. Here is the matching tml file for the class above;

<html t:type=“layout” title=“Peoples I Might Know”

t:sidebarTitle=“Current Time”

xmlns:t=“http://tapestry.apache.org/schema/tapestry_5_1_0.xsd”

xmlns:p=“tapestry:parameter”>

<!– Most of the page content, including <head>, <body>, etc. tags, comes from Layout.tml –>

<t:form t:id=“search”>

<t:textfield t:id=“searchTerm” validate=“required” size=“20″/>

<t:submit t:id=“searchPeople” value=“search”/>

</t:form>

<t:grid source=“persons”/>

<t:form t:id=“clear”>

<t:submit t:id=“clearPeople” value=“clear”/>

</t:form>

</html>

Yes, it’s that bloody simple. The first form fires the “onSubmitFromSearch” method (with a little bit of validation, done in javascript) and the second method clears the list. In-between, there is this <t:grid source=“persons”/> business, which, if the ‘persons’ variable in the page class is populated, will show a list of its contents!

To test our app, we can use the command-line ‘mvn clean jetty:run’. When Jetty has run up, then we can point our browser at the web app: http://localhost:8080/sample-webapp/Persons, and with any luck we will see the following:

Now, by default, Tapestry creates the web app to use the template design as shown above. Of course, you can make it look completely some other way – or even use a totally different template if you want. The page template is just a component that’s included.

So if you enter in a search term and click the search button, you’d expect to see a result like this:

As you can see, the list is automatically populated with the details returned from the service method (in this particular instance, this are just a canned response, but normally of course they’d be the result of a database or a web service call of some type).

When you click the clear button, the list is cleared as you’d expect:

Now, here’s a really powerful feature I find in Tapestry. Let’s say our automated acceptance tests assert that when the ‘Clear’ button is pressed, the Search box as well as the persons list is cleared. What Tapestry allows us to to do, is keep Jetty running, edit the files in the IDE, and re-run the tests against the running app without restarting! We edit our method:

    Object onSubmitFromClear() {
        persons = Collections.EMPTY_LIST;
        searchTerm="";
        return this;
    }

In this case I’ve also added a bit of space around the clear button in the template as well:

And now our failing test will pass.

Editing files like this in the running web app only works for Pages and I think Components. If I had to change the service or model classes I’d have to restart, but “mvn jetty:run” isn’t very a heavyweight process.

As I said above, Tapestry’s not perfect: it’s major flaw is the poor documentation. Convention-over-configuration is easy to grok – if you know the convention. If you don’t and the documentation doesn’t tell you, and you can’t find a sample of what you need, it can be very frustrating. There is an excellent user list though.

Its other major flaw may be the stability of the API. Tapestry 5 is different (and incompatible) from Tapestry 4 is different from Tapestry 3. But so far I’ve used it on a couple of projects and I’m really enjoying it. I really hate the oodles of XML boilerplate and massive amounts of configuration found in Spring and I find myself somewhat reluctant to use it unless I really have to. Tapestry solves for me a range of different problems and mostly it presents a very elegant way to create a componentised web application. You might like to give it a try.

Attached is the sample code used above. It took me about 10 minutes to write (far shorter than it took me to write this blog entry!) - sample-webapp.tar bundle

A few people seem to be sweating about MySQL. It would not be stressing about MySQL too much. It could get spun off, who knows. It’s even possible, as some commenters on Techcrunch say, that Oracle might use the free MySQL offering to hammer Microsoft’s database market from the bottom-up

It’s more touch-and-go in the app-server market (what Oracle likes to call ‘middleware’) which is already suffering a little from Oracle’s transition over to BEA Weblogic from its older products. Sun has excellent products in that area, i.e. Glassfish. I have used all of these products (plus Websphere and JBoss) and Glassfish is easily the nicest (in fact I would say it’s the best app-server I’ve used, apart from plain old Tomcat).

And what will happen to Java? Of course, Oracle wants Java, that’s part of the reason they are buying Sun in the first place (as well as their hardware business). But will IBM play along now it’s most critical competitor owns Java (and IBM has previously bet its software integration farm on the Java stack)? And what of the JCP?

Even more worrying for some, is what will happen in the IDE space. Of course, I’m a confirmed Eclipse man, but it is always a worry when competition is reduced. What will happen to Netbeans? And dare I say it … JDeveloper is fairly horrible compared to Netbeans but will that save either of them? Oracle’s got a lot invested into it’s tooling, which all runs on JDeveloper. As much as I prefer to just write code, that use wizards, Oracle does seem to have at least some customers in that area. Oracle’s development model tends to focus a little too much on magic wizards. From the IDE-JDeveloper-to-the-app-server single click-to-deploy, drag-and-drop to create-the-control, easy-peasy wizardry, which I hate, because I think it gets in the way of craftmanship. It’s the Bunnings Warehouse of software development. Show me the command-line Oracle! you should remember, that Sun is most definitely a company built around the Unix shell prompt. Even with Oracle middleware, it’s definitely possible, I’ve just been working on command-line deployment automation for their older Orion-based app-server, they just don’t like to promote it.

I’m hoping that Sun can reverse-ferret them on that score and teach them about Ant, Maven, /bin/sh and the goodness of installations that are as simple as tar xcf in the spot you want to install it.

My case-in-point du jour is Oracle’s new Rich Enterprise Applications site and the associated technology. On that site you will find a flash presentation of their technology around delivering and I quote:

multi-channel capable applications that deliver desktop quality, highly interactive user experiences that are pre-integrated to enterprise class server technology.

That sounds pretty good, might be of interest to me. Lets investigate further. Now, a lot this technology is built on top of standards, like JSF, and other commonly used technology like AJAX. The earlier version of the ADF component technology was donated to Apache where it lives as the Apache Myfaces Trinidad project. The new components are built on top of these open sourced components. So far, so much is OK.

But, the kick in the tail can be seen in even just in the quoted sentence above: “pre-integrated to enterprise class server technology” – they certainly don’t mean Tomcat, right? The point of the presentation is induce lock in. However I’m still willing to cut them some slack on this particular point. Of course, if you are thinking about using Oracle’s JSF technology, you probably are already committed to Oracle’s middleware technology anyway – and Weblogic is a perfectly good application server, if you need such advanced capabilities (and whether you actually do is another question for another day).

But what absolutely does my head in, is the insistence the only way to develop for this technology is to write it using their proprietary, non-standard developer technology, aka JDeveloper. This is the point where I have to say, I’m sorry Oracle, but … FAIL. I actually don’t even care if your IDE is even the world’s best! Or even if it comes as an Eclipse plugin. My beef with this sort of IDE integration is the total mind-set that it embodies. The IDE might have all the drag-and-drop interface-building mojo in all the world, but the code it produces will never, ever, stand up to a hand coded model driven off the target domain, using modern agile development techniques that are designed to ensure the most cost-effective, best-quality development delivering timely business process transformation (aka value) that the enterprise prioritises and chooses. Just look at the ADF FAQ on the Oracle Wiki. Half the questions are related to the IDE. Here’s a tutorial style introduction that just assumes JDeveloper as the default way to get access to the Oracle JDF components.

No, no a thousand times NO!

The biggest insult is the way that developers are treated as mere commodities in this whole process. They are just the warm bodies occupying the chair driving the interface. Much like the continually failed dream of developerless development, their primary skill is seen as not “being a programmer” but “being a driver of a software package”. If I said, my financial skills and experience include 10 years Excel and 5 years MYOB – I can haz CFO jobz now? NO?! Why not? It’s insulting, that’s why not.

Software development is a profession, or at the very least a craft, and software developers should be able to choose their own tools. Obviously, some agreement has to be made organisationally, or at least across a single team or a whole project, to use a certain common toolset. Here’s one possible checklist:

We’re writing a Java app, check. Deploying onto Weblogic, check. Database is Oracle 11g, check. We have Scrum as our process framework, check. TDD is in our list of favoured engineering disciplines, check. Maven is our chosen build tool, check. Using Subversion for version control, check. Hudson for a continuous integration server, check. Writing the interface with Oracle ADF, check. Using Crossfire to deliver our web services, check. Spring for application wiring, check. Hibernate for persistence, check.

Each of these things has to be agreed upon at an architectural and organisational level (and obviously I don’t mean to say that each of the above is the only choice for each type of item, I’m just giving examples).

You should note one very deliberate omission from that list, the IDE. Architects might impose a certain application server, a particular approach to application layering, and so on. The developers – at least the senior ones – have to choose and agree, for example, on whether to use Ant or Maven, CVS or SVN, Cruisecontrol or Hudson. But what they do not do, is force developers to favour Eclipse or IntelliJ or Netbeans or JDeveloper. What is the point of taking someone who is super-productive with a particular IDE’s keystrokes and then forcing them to spend six months learning a whole new set, a new environment? That’s not productive! A developer needs to have the IDE get out of the way, become second nature – they need to be thinking about code, not where the menu is for the rename-method refactoring because they are unfamiliar with the keystrokes of the unfamiliar IDE. Keystrokes are muscle memory. You just think, rename this method and your fingers are already on the keys you need to press, without breaking your all-important flow. That’s how a developer is as productive as they can be. A productive development team should have a mix of IDEs (I’d propose that this would be statistically in proportion to the usage patterns of the common IDEs). Then we have the specific example of JDeveloper. I won’t even go into it except to say I’ve never heard a developer do anything but curse it, or maybe laugh at it. But it’s the laughter of the condemned man.

It’s something that vendors really need to get serious about. Rather than presenting me with a seamlessly integrated vision, from developer’s screen, through application server, to user’s screen, their technology presentations ought to think discrete. Each piece of technology ought to live and die on it’s own merits, something I’m sure Oracle’s ADF is perfectly capable of – without JDeveloper. If it’s not capable of being developed productively without JDeveloper, it’s already in the reject pile as far as I am concerned.

This essential information needs to be right up front. Don’t bury the information a thousand web sites deep in your developer portal. Put it right there in the first presentation! Separate the selling of the developer tooling from the selling of the user presentation frameworks and the server technology. Show me how I integrate Oracle ADF into TDD, into continuous integration, into the IDE of the developer’s choice. Because if you don’t, you’ve already lost me as a potential customer. This is the real future of software development. Look at where the major thought leaders of software development are taking development – open source frameworks, inversion of control, TDD, Domain Driven Design, Domain Specific Languages, Agile Methods, delivering real process transformation and business capability as early as possible – not drag and drop with wizards. FFS.

I might agree that for simple enough systems, it could end up being that in fact the service layers end up with PersistenceManager logic directly in their methods. But then I might say a language and framework like Ruby on Rails could suit your application better in that case. For any sufficiently complex Java-based enterprise system, a POJO-style data persistence layer naturally evolves, in order to reduce as far as possible the responsibilities the service layer must carry out.

As mentioned by commentators on his blog, the discipline of test-driven design is usually  a strong driver of having a separated set of components that deal with persistence and other data concerns. Not just from the issue over mocking the PersistenceManager, which is certainly a strong driver, but also from design principles.

Service layers marshall requests from clients, and decompose them to call various component layers. Data access layers typically form part of these component layers. A TDD discipline means these data access layers usually end up with more than just CRUD operations.

Furthermore, as a matter of discipline, in any sufficiently complex service layer I prefer not to work directly with database abstractions (i.e. entities that are modelled on database tables and their relationships) but on first-class objects designed to fit directly with the modelled domain. I want those objects to have rich behaviour – the service layer is actually part of their behaviour. I want to start the design of the system by directly modelling those actions, e.g.

passenger.at(origin);
passenger.orderJourney(dispatch, destination);
Vehicle vehicle = dispatch.sendVehicle(origin);
vehicle.pickUp(passenger);
vehicle.arrives(destination).dropsOff(passenger);
vehicle.notify(dispatch);

I do not want these objects to know about database semantics. That is a responsibility of a separated persistence layer, which we may for convienience sake, called the DAO. Even if internally we end up with;

class Passenger {
  at(Location origin) {
    this.origin=origin;
  }
  call(DispatchCentre dispatch, Location destination) {
    dispatch.passengerOrder(this, destination);
  }
}

DispatchCentre at this level, is a service, but we will model it as part of the domain of passengers, locations, and vehicles. Behind that service, it will have a database layer with the semantics as to how it saves and loads the data concerning passengers, locations, and vehicles. I don’t necessarily want the client objects (e.g. the web action) to have to deal with JPA annotated objects. They just deal with objects as much as possible. And I don’t see there’s anything served with developing possibly complex persistence logic directly in the DispatchCentre. In fact, taking a top-down approach, I know pretty much I won’t do that.

Only after I’ve done this initial part of the modelling, will I be concerned as to which of these components can be directly JPA or Hibernate annotated, which are Spring-injected services, which are Session EJBs, and the like. I think starting your modelling from the point of view of having a persistence framework like JPA or Session EJBs or any other ‘archtectural artifact’ leads to an unnatural design.

I am now a contractor for a large government organisation and as part of my role there, I get to participate in these workshops. Trust me, a fruitless exercise. People bickering about their favourite frameworks, why to use Axis over Spring-WS, why everyone should use maven style dependency management, maven is the devil, etc, etc.

Well, this has been a problem in Java circles for a long time now. I think we are learning to live with it. But what I don’t clearly see is its full relevance to Java “architecture”.

In my view, architecture is an independent set of constraints on the selection of the requisite frameworks; but the selection of the frameworks isn’t itself  an archiecture, nor necessarily an architectural task. In my view, achitecture is the view that says, “this is a web service” or maybe in more detail: “these are five components ‘C’ {Ca, Cb, Cc, Cd, Ce} having responsibilities ‘R’ {Ra, Rb, Rc, Rd, Rd} and they communicate via an asynchronous messaging bus in message formats ‘M’ {M1, M2, M3}”. The actual selection of the frameworks to accomplish this just should not be charged to a committee – ideally the developers should do it, with appropriate guidance from the relevant architect (who should be on the development team anyway).

Of course this proliferation of frameworks and standards to get this stuff done can lead such committees up the proverbial garden path. But, and without being privy to the full scope of the particular organisation Simon refers to, the essential problem I see here is the function of the committee – it’s just compounded by the additional issue of choice, and for some, those choices have inevitably become religious, making the said committee even more dysfunctional. Some would argue the proliferation of frameworks is something that makes Java stronger. There’s a marketplace in frameworks. But we all know about design by committee, don’t we?

That said, a lot ‘standards’ (even the JSR standards) are indeed irrelevant and out of date, as I pointed out only a few days ago.

In closing his blog entry, Simon leaves us with a question:

In closing, do you think JEE can be saved from becoming the next dinosaur?

Well, that, of course, is the million dollar question for a Java programmer isn’t it? Myself, I just prefer to get down and dirty and into the writing of the code – the business objects – that encapsulate the domain at hand. If we can’t just naturally select an uncontroversial framework to enable some segment of the desired infrastructure, then the best thing I find is to use your process, or more appropriately, your design, to just route around the controversy. But the best position is just not to be in that position anyway. Unless the chosen framework absolutely prevents you from doing something you absolutely need to do, you’re not quite in a pile of stinking do-do just yet.

And if your architecture committee is bogged down in arguing about framework selection, then I’ll tell you your real problem is is your committee, not the frameworks.

Sun Microsystems is getting some love from Wall Street after its sales and earnings topped estimates, as detailed by Bloomberg. Software sales jumped 21 percent year-over-year.

What is fueling the growth? The same thing that Sun CEO Jonathan Schwartz has doggedly said would lift Sun’s fortunes again: open source.

via Open source makes Sun’s quarter shine | The Open Road – CNET News.

If this approach is validated going forward the next year or two, will it change the software industry radically? Quite possibly. Remember this is not a five-man company selling training and consultancy services leveraged off it’s founders’ successful open source software effort, nor even a much larger company set up solely to market and sell services and support contracts based around packaged open source distributions. Sun is a big vendor, which has had to turn its ship of business around almost 180 degrees to get where it’s going now. And maybe, just maybe, it’s on a winning course now.