The technology world has just lost another giant, though one without the towering public persona of Steve Jobs.

If you’re not actually a programmer, you’ve probably never heard of Dennis Ritchie. But the vast majority of software you use was built using a tool that he originally designed, and the rest by tools that very liberally sample from his.

The “native language” that the central processor in a computer understands is an ornery beast. For one thing, back in the 1970s every two-bit computer company (if you’ll pardon the techy pun) had its own native language; these days, there still remain two very common ones, and dozens of less common examples out there. More importantly, it’s almost incomprehensible, even to most programmers. Take this little snippet, part of the preliminaries to a very simple program that just prints the message “hello, world” to the screen:

_start:
        mov    eax, 4
        mov    ebx, 1
        mov    ecx, str
        mov    edx, str_len
        int    80h

Writing long and complicated bits of software with such unhelpful notation is extremely slow and error-prone.

“High-level” languages, that allowed the logic of software to be expressed in more compact and readable notation, had existed since the 1950s; Grace Hopper was responsible for one of the first. Over time, more and more of the scientific and business software run on the large computers of the era was written in FORTRAN, COBOL, and other high-level langages. However, the “operating systems”, the software plumbing that joined those applications to the hardware, was invariably written in the machine language of specific systems.

In the late 1960s, Ritchie, working with Ken Thompson at Bell Laboratories, hacked together their own little operating system for an obsolete computer nobody was making use of. It was small, but it worked, and was one of the first practical systems to support “timesharing” – the ability for multiple users to run multiple programs simultaneously and interactively. Fairly early on, they had another brainwave; they would rewrite as much of the system – which became known as Unix – as possible in a high-level programming language, to speed development. But first, they needed a suitable high-level language. The resulting language, an evolution of earlier languages entitled BCPL and B, was called “C”.

Both C and Unix were raging successes, partly because of their inherent strengths. The use of C allowed Unix to be “ported” to many different computer systems, a process that continues today as its spiritual successor Linux, written in C, runs on everything from IBM mainframes (and the amphormous “Googleplex” of Google’s servers which, reportedly, draw 240 megawatts of power), to virtually every smartphone on the planet (the iPhone’s operating system is not Linux, but it is also a derivation of Unix and substantial parts are written in C). They also had the good fortune, as with the Internet and World Wide Web which owes so much to both, that they gradually escaped the crush of intellectual property law to become part of the intellectual commons of the field.

Almost as important was the elegance and economy that Ritchie, along with Brian Kernighan, brought to teaching the language. Their textbook The C Programming Language remains the best programming language textbook ever written, in my view, and the one that I still strongly recommend to my students.

Much of the Windows operating system, and Mac OS X, are implemented in C. Those parts that aren’t, are implemented in computer languages directly derived from it – C++ and Objective-C. Most of the software that runs on those systems is also written in C or its successor languages. And perhaps the most pervasive “new” high-level language of the last 20 years – Java – retains so much of C’s “look and feel” that it often takes a second glance to tell which language a piece of code is written in.

Neither C nor Unix were by any means perfect. While some of its design faults have been eliminated in its successors others remain, and will likely continue to bamboozle neophyte (and, all too often, experienced) programmers, for generations to come. But there was so much he and his colleagues got right. Fate did play its part, but there are very good reasons that generations of software developers not yet born will express themselves in notations largely based on Ritchie’s.

RIP, dmr.

via Dennis Ritchie: The Shoulders Steve Jobs Stood On | Wired Enterprise | Wired.com.

It all started when I needed to start the Transaction out in the view, i.e. as soon as the resource is opened on the HTTP side (rather than when the database service layer is called). I’m using JPA for a number of reasons;

1. I need to access multiple databases each with a different schema (this means different connections)
2. They need to have XA transactions.
3. I’d like the transactions managed by the container (JTA).

JPA provides all these things easily without all the complex Hibernate.xbm.xml mapping files and what-have-you.

The trick to starting the transaction with the web session is to use spring’s OpenEntityManagerInViewFilter. Unfortunately I was using  PersistenceAnnotationBeanPostProcessor to manage my multiple persistence units. The filter wants to know what EntityManagerFactory it should bind to, and fair enough. But with my minimal configuration, there was no addressable EntityManagerFactory!

The solution was to stop the PersistenceAnnotationBeanPostProcessor from doing the JNDI look ups and bind each PersistenceUnit into the Spring context with a manual JNDI lookup;

  <tx:annotation-driven />
  <tx:jta-transaction-manager />

  <bean id="pabpp" 
   class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor" />

  <jee:jndi-lookup id="onePU" jndi-name="persistence/onePU" />
  <jee:jndi-lookup id="twoPU" jndi-name="persistence/twoPU" />
  <jee:jndi-lookup id="threePU" jndi-name="persistence/threePU" />

then, in the web.xml the ViewFilter could be bound explicitly to the required persistence unit, in this case “onePU”. A fuller explanation can be found on my spring source forum post here; http://forum.springsource.org/showthread.php?115844-OpenEntityManagerInViewFilter-with-JPA-PersistenceAnnotationBeanPostProcessor

Overall, this is now quite an elegant solution.

UPDATE. More documentation of the solution and the various configurations at the Spring forum here: http://forum.springsource.org/showthread.php?115587-Example-for-using-two-databases-w-Spring-amp-JTA-transaction-manager&p=383432#post383432

Our system has generic components which “pass off” specific knowledge of specific data sets to specific processing components, where there are not a fixed set of these specific components available to all the generic components. In classic OO design this is achieved with interfaces and many design patterns like the Visitor Pattern. In our design we’ve replaced much of the complexity of the interface with REST-based architecture. Our application has become far more modular (despite its run-time complexity). More importantly, we are at the the point where we can easily start delivering the advanced features that the business has now specified for us without introducing a massive bloat-o-burger one-size-fits-all set of “common data definitions” that are the union of all possible sub-system functionalities.  Each sub-system can have its specific features required for that data set and still satisfy the handful of generic actions it is required to support. Each of these specific features can be developed in isolation. It’s a big win for us.

The problem is simple. In Weblogic, if you cluster a logical topic (or a queue) across two machines, underlying this you’ve actually got two uniquely named and non-cooperating physical topics, one on each member of the cluster. A topic message that is sent to the logical topic is delivered (copied) to both physical topics. Now, if your logical topic listener is also clustered, you’ve got, in effect, two physical topic listeners on two different, and non-cooperating, physical topics. The topic message is then always delivered twice – even if both listeners have identical durable subscription ids (i.e. indicating that they belong to the same logical component, which should only get the topic message once). This terrible architectural blunder occurs because there is physically underlying your logical configuration, totally separate, non-cooperating, uniquely-named topics on each of the cluster members, with what amounts to a JNDI name service hack in the middle to spray the load about your cluster. In other words, it is load balanced, not clustered. JMS Queues are the same – however its just that the implicit side-effects of the queue semantics – which ever only delivers to a single consumer even if there are many configured – means it isn’t impacted as much. Because of that semantic, a queue message is only delivered to one of the two physical queues (i.e. again, it’s load-balanced) so only one of your two clustered queue listeners gets the message. And there’s a terrible time-out based hack to redeliver the message to another member of your queue cluster just in case there isn’t any listener instances on the cluster that the message ends up on.

N.B. Coherence does not, apparently, have this massive architectural fail. If you need to do clustered fail-safe broadcast messaging in an Oracle Weblogic environment, do yourself a favour and use Coherence.

For a start, the documentation is merely a series of outlines of brief hints. One measly section.The laughably short Spring 3 doco section 13.5.1.4 “Dealing with multiple persistence units” conveniently omits  the transaction manager configuration from the example. The problem appears to be that the JPA transaction manager only (and compulsorily) deals with one entity manager factory. And an entity manager factory only deals with one persistence unit. So therefore you have to have two JPA transaction managers. Which means the two persistence unit transactions won’t co-operate properly even though they may share the underlying datasource and are configured through the single persistence unit manager bean.

The above scenario is totally trivial in an EJB 3 container backed with Hibernate, or Eclipselink, as the provider. About one-quarter of the configuration. If I have to use a JTA transaction manager obtained by JNDI lookup from the container, to run a transaction across two JPA persistence units which share the same underlying datasource, why the hell am I using Spring in the first place?

All I wanted was to isolate one set of db transactions from another, use JPA persistence units so the two sets of tables could live easily in two different schemas, and use the annotations to kill the fragile AOP regex-like-but-not-like-regex class and method pattern-matching jiggery pokery from the Spring configuration.

My god, this is such a stonkingly non-trivial forest of horrible configuration and a trial and error morass of filthy swamp miasma … an equivalent EJB3 backed by Hibernate JPA set up is, by comparison, an almost effortless task. I can produce a unit-tested all singing all dancing multiple persistence unit JPA app running in an EJB 3 container with XA datasources using Hibernate as the JPA implementation in almost no time at all. Getting the same thing with Spring (oh, and running some tests with Jetty inside Maven) is like having all your teeth pulled while you’re coming down from a three day methamphetamine binge. Getting Spring to run with multiple JPA persistence units on the same JDBC connection (without XA, across the same database connection) with a transaction propagation of REQUIRES_NEW on the entry point of one of the units is a hair-pulling, beard-greying, head-desk banging, co-worker punching, drunken ranting, blood-pressure raising experience of pure horror. Which apparently doesn’t end.

Die, Spring, die. Can’t come soon enough as far as I am concerned. It’s more evil than Oracle. At least you know with Oracle you’re in for an un-lubed hard and fast backdoor job from Ellison with no reach-around before you even unpack the box. Spring is like a beautiful young sexy soft-porn film that turns into some relentlessly horrific succubus-filled horror film half way through.

Seriously, its enough to make one pine for bloody Weblogic. Spring is the new EJB 2.1.

First the technologies and the development methodologies.

So the first company uses a very Waterfall process and the integration platform is SOA. We’ve managed to build, in the middle of this, a small and focussed Java component that uses JMS in and out to avoid building horrible horrible BPEL or BPMN etc. But at either end, there’s some SOA bits to manage the integration with the “legacy”. So we’ve got this great bit of software, does amazing things, delivers real value to the business, which they want to put into production asap, but at every turn we’re hampered by the organisation or the technology platform. Its not so much the technology platform but the waterfall process the company has put around it. I’ve been told an internal wiki article detailing all the JMS  configuration is not  acceptable and the detail had to be in a Word document attached to an email requesting a not-production server configuration change. Word documents attached to emails are apparently far more “controllable” than a wiki with strong authentication protocols and history details in this world-view. Naturally I cut and pasted my wiki configuration detail into a Word document, spent a morning formatting it, and even added a link to the wiki before attaching it to an email. Acceptable process; configuration delivered. SOA and waterfall go together to make software development hell.

The second project has its many issues but one thing it does not have (for the components I have designed at least) is any SOA. It is all REST all the way down. There are multiple server-side-only components that all communicate to each other with REST over HTTP. Some of the data passed between servers is (or soon will be) JSON. The user interface is about to be delivered in two parts – one through actual REST-inspired web page requests to get the HTML and the other, via JQuery running on the web pages to actual JSON over REST services. These services will also call the REST services on the other components of the system (no database at the UI level). In fact many of the same JSON documents will be used to communicate from server to server as from ui to server. We are using Jersey for the REST. The process is Agile. The organisation hasn’t delivered such a large project with Agile before, and while I’ve been away on the other project there’s been some loss of focus. But despite some critical moments we had last week there’s been a renewed commitment to improving the engineering and project processes to achieve high velocity and good success. Its not perfect but it feels like the senior managers want the agile process to succeed. So, REST and Agile go together to make developer success.

Now, to the organisations. See if you can match the organisational description to the project style.

One of these two projects is a really fascinating piece of software in the transport industry (I can’t say more than that) which is integrating a number of “old” software solutions and creating some really sexy new features that the customer wants and loves. It is across what Eric Evans calls the “core domain” of the business. Not only is it a relatively short piece of work, but it will deliver high value to the business. It also enables a bunch of even higher value business services in the future. Things that directly expand their capabilities they can offer to their customers (which will be many of my readers) and do other related sexy things core to the business. The business salivate over it. It buffs their shine. They want it.

The other project is a little less flashy. The product is a rather niche product but it’s purpose is around the environmental effects of carbon emissions so at it’s core is the mission “save the planet from CO2″ — a pretty high value mission you’d think. However the actual functions of the software product are fairly mundane – based around helping large organisations capture and control data about their carbon emissions, because many of them have been required now to report these numbers to government agencies for a number of years. It has customers, all of then big companies or government agencies, and all the future prospects are similar sorts of organisations.

Which one is which? Which one uses REST/Agile, and which one is the SOA/Waterfall project? Do you think you can guess?

Would you be surprised if I said that the “sexy” transport project is the first one (SOA/Waterfall) and the second one is the REST/Agile project?

Well, nope.

Especially nope if you hope to use clustered broadcast messaging (i.e. Topics)  to multiple connected clustered application receivers. Unless you cope with multiple versions of the same message in your application code.

Funny thing is, they already have this product that can do clustered broadcast messaging well proper, it’s called Coherence. But if you need the JMS API, not so suitable. However they could use this product to underpin their crappy 10 year old clustered JMS implementation in Weblogic. Do you think they will do that? Anyone want to open a book on it?

Overview

OSB installation in a development environment consists of a completely separate Weblogic instance and yet another ‘special installation’ of Eclipse. You can’t use existing Eclipse installations. Nor is it recommend to use one of the five other existing Weblogic instances that Oracle products thoughtfully demand their own copy thereof (SOA Suite, I am looking at you).

Installation process

This is the procedure for installing OSB development environment.

As network firewall policy prevents the downloading of files, you will need to use installation media located within <companyname> network. Take the files conveniently stored on network server <location redacted> and copy them into a directory on your local computer. This directory is hereafter referred to as ‘installation files directory’ below.

Install Weblogic

1. Go to the installation files directory.
2. Run the executable file wls1033_oepe111150_win32.exe
3. This action launches the Oracle Installer for the Weblogic instance. It takes some time to run so contemplate the meaning of your life for a few minutes while it does. Maybe get a cup of tea. Stretch your legs. Think about lunch or your next holiday. File a support request for more memory. You should get at least 8GB.
4. Okay that’s it, you’ll finally see the Oracle Weblogic 10.3.3.0 installer. Think about the fact that Oracle sees fit to put “instructions” on the first screen telling you what the “Next” button does. This is advanced JEE server technology you’re installing, and potentially, Oracle think you don’t know what the “Next” button does. Maybe this reflects the level of experience inside Oracle, or perhaps it is indicative of the depth of respect in which Oracle holds their customers.
5. Press the “Next” button. The next screen you will see  is very important. If you fsck it up you’ll have to uninstall and start over. It does not have any instructions, but that’s OK, we’ve done it here in this wiki page for you. Pay attention.
6. DO. NOT. ACCEPT. THE. DEFAULT. Especially if you installed SOA Suite or some other Oracle product before you installed this one. Double especially if you still want that product to work.
7. SELECT “Create a new Middleware Home” (THIS IS VERY IMPORTANT)
8. TYPE a name of an Appropriate Directory. I used “C:\Oracle\OSB_Middleware”
9. PRESS The “Next” button. Use the “Back” button if you need to see the instructions about how to use the “Next” button that Oracle conveniently provided for you on the first Screen. No, don’t do that. If you do that you’ll probably have to do this proceedure again and it will just make this entire experience last longer than it needs to. This is not a recommended practice.
10. The next screen demands that you give Oracle your Email Address to get “security updates”. It also wants a thing called your “My Oracle Support Password” (I suspect this is what might have been known as Oracle TechNet). As we both know, the best possible security measure is not to give out your password to strange programs that demand it.
11. As I planned on giving them my <workcompany> email, and as my Technet sub doesn’t use that Email address, I also unchecked the “I wish to receive security updated via My Oracle Support” check box.
12. Are you Sure? YES I’M VERY SURE. I would like to be “ignorant of security updates” and also Oracle spam. Ignorance is Bliss.
13. Look at that, I can’t give them my email address after all, what they really meant was “type your Oracle Support user id”. Press “Next”.
14. Now you have to choose whether you want a “Typical” installation or a “Custom” one. I chose “Typical”, which, being an Oracle installation, I expect to require an 8-core 64GB RAM 2TB SAN SSD -based supercomputer with a external 4-way Oracle RAC for configuration (“infrastructure”) DB in order to have enough grunt to service about 3 requests a minute. Press “Next”.
15. There’s a lot of choices here about the various subdirectories under the Middleware Home Directory that you created further back. I recommend accepting the defaults, but you can probably cause yourself countless of fun trolling on the My Oracle Support forums as you get ever-more-desparate for a solution to a very obscure problem that the phone support have no idea about and that was likely caused by you mucking about these defaults, causing the support personel to simply recommend you to reinstall the product, if you really feel the need to change them here.
16. If you can remember the instruction about the use of the “Next” button at step 4, then Press “Next”.
17. Now you can choose whether you want to put the shortcuts for “All users” or just you (“Local user”). If you are the BOFH I recommend “All Users”. As this is the default, we can all safely assume that the BOFH works for Oracle and is now responsible for designing their installation processes. Accept the default, and press “Next”.
18. The next screen is a summary of what you’ll be installing. You can also select each item and see a summary of what it does. Ponder the mystery of Oracle, and press “Next”.
19. Keep pondering that mystery while Oracle Weblogic Server 11g Release 1 (10.3.3.0) is installed. It takes a little bit of time. While it does that, you might to book that Holiday, get another cup of tea, or chase up that support request for the additional RAM you’ll be soon needing.
20. Congratulations! Installation is complete.
21. I opted to leave the “Run Quickstart” option checked.
22. Press “Done”. There are no onscreen instructions for this button.
23. Quickstart will run. It’s just a link farm.

Install Oracle Enterprise Pack for Eclipse (OEPE aka “Special Eclipse”)

1. Go to the installation files directory
2. Extract the file oepe-galileo-all-in-one-11.1.1.5.0.201003170852-win32.zip … I used 7zip and made sure to put it into a subdirectory of the installation files directory.
3. It’s kind of pretty big, takes a cople of minutes.
4. Just like a regular version of Eclipse, once this is unzipped, it’s installed. However it’s not a regular version of Eclipse. It is a “special” Eclipse that went to “special” school.
5. Although this directory can probably live anywhere, it’s a good idea to copy this directory into the new Oracle Middleware Home that you created when you installed Weblogic. Look I really have no idea why this is the case, however, it’s not good to anger the Oracle by using you regular development directories. ORACLE_HOME sweet ORACLE_HOME it is then.

Install OSB and OSB Dev Tools

1. Go to the installation files directory.
2. Extract the file ofm_osb_generic_11.1.1.3.0_disk1_1of1.zip … again I used 7zip and made sure to put it into a directory underneath the installation files directory.
3. Enter this directory. Enter the directory “osb” that will be created underneath it. Note that even though the file said “disk1_1of1″ in the file name that underneath here there’s two directories, Disk1 and Disk2.
4. Go into the directory “Disk1″
5. Run the executable file “setup.exe”
6. A DOS window opens which asks you for the location of a JRE in order to use Oracle Universal Installer. Probably. Exactly why the Weblogic installer didn’t need to know where the JDK was, I do not know. Probably it used a sensible installer rather then the Oracle Universal Installer. I don’t ever think I’ve ever seen an machine with two Oracle installations on it that didn’t also have two or more installations of the Oracle Universal Installer also installed on it. Its name perhaps means that it installs itself universally, rather than it is a product which has a universal use for installing other software. Ponder the mystery of the Oracle.
7. A JRE will be located in the original Oracle Middleware Home that you created when you installed Weblogic. In fact there’s at least two (Sun JDK and JRockit). Use the Sun JDK. For example, my value for the JDK was “C:\Oracle\OSB_Middleware\jdk160_18″. Press Enter.
8. Now the Universal Installer will actually attempt to install something. It says “You are about to install the Oracle Service Bus (OSB) and may install the Oracle Service Bus IDE and Oracle Service Bus Examples (OSBE). Before proceeding, make sure that you have installed and configured Oracle WebLogic Server 11g. If you want to design OSB applications in Eclipse, make sure Oracle Enterprise Package for Eclipse (OEPE) is installed.” Which is all true if you’ve been following this guide.
9. Press “Next”. There’s no instructions in this program for the use of the “Next” button. Someone ought to file a change request for that.
10. Now you can choose whether you want a “Typical” installation or a “Custom” one. Typically, choose “Typical”.
11. Press “Next”. I tried looking for online help here about the use of the “Next” button but I did not find anything.
12. It does a prerequisite check. It should pass, and if it doesn’t, you are probably screwed. If it does, you will be able to Press “Next”
13. At this next screen DO NOT ACCEPT THE DEFAULTS.
14. Choose the Oracle Middleware Home that you installed the Weblogic into at the first part of this installation procedure. E.g. I chose “C:\Oracle\OSB_Middleware”.
15. Once you do the previous step, ff you followed the instructions for the Special Eclipse (OEPE), it will have found it automatically. If not, choose the location where you installed the Special Eclipse (OEPE Location). For example, my value was “C:\Oracle\OSB_Middleware\oepe-galileo-all-in-one-11.1.1.5.0.201003170852-win32″
16. Press “Next”. Did you know that the Oracle at Delphi was a priestess called the ‘Pythoness’ who answered your question using gibberish verse. A Male Attendant of the Pythoness interpreted her raving mad gibberings and told you what they meant. For a fee. Does this sound familiar?
17. Review the installation details. When you are sure they are correct, press “Install”.
18. OSB will now install. It takes a little time so run those errands, go to lunch, get a coffee, dream of the Holiday you just Booked. I’d tell you to install the new 8GB of memory that you ordered before which surely has arrived by now, but that would mean turning your computer off. Best to wait until it’s finished then.