Skip to content

New software, old process, big mistake

Its very common for software developers to be asked to build some software that is a straight port of an old software package, or to faithfully model (i.e. completely identical to) an existing process that the customer has. This is a huge mistake – try to avoid these projects. I hold that if the customer wants software, either custom developed or “off the shelf” purchased from a vendor, they are already changing their business model (aka their “process”). It’s the worst possible to thing to build or buy software and just model what is already done (perhaps it is actually impossible). As an senior developer or architect, my riposte to these requests is always “well don’t spend any money and just do whatever it is you do now”.

I don’t hold that software makes an existing business process “efficient” at all. Rather I think software makes possible a new process, which should be more “efficient” in terms of money gained less dollars spent – but its a new process, not the old one. In effect, new software creates new business opportunities. New software will only make an existing, unchanged process, less efficient, if a new business process is not designed along with the new software. If the business just wants new software without changing “what they do” they are wasting their money, IMHO.

Of course there is the possibility (probability?) the business doesn’t actually understand what it is they actually do anyway. This is not an uncommon position for many businesses that are happy to cruise along in neutral making some marginal profit on some marginal activity. Usually these businesses are also found to be beating their workers with sticks (usually only metaphorical ones unless they ‘offshore’ their operation to countries where killing your workers is just a part of ‘Business as Usual’. Typically they hold that marginal process can be made ‘better’ simply with just more exhortation (or threats) to greater and greater efforts at a totally demoralized (if not actively hostile) workforce, but I suspect that’s a story for another day!

Glassfish is doomed in the ‘department’

There’s been lots of discussion the past six months about the fate of MySQL under the ownership of Oracle. Now that the purchase of Sun is complete, I’m much more concerned about the fate of the excellent JEE platform Glassfish. For example some people think that superior technology will prove to Oracle that Glassfish is worth pursuing (see the comments on this dZone thread about Kenai.com).

The problem for Glassfish, as the second sentence of this ServerSide article states (see it straight from Oracle’s mouth here, and see also here) is that Oracle view it as being used for “non-mission critical department apps”. Glassfish’s superior technology (or otherwise) just doesn’t come into it. It’s not a factor (as it rarely every is).

Not so long ago Oracle spent a big wad of money acquiring an app server (Weblogic) and then a stack of more money porting all its other products into it and branding the resulting mess platform “Oracle Fusion Middleware 11g”. Now not only do they have their third app server (OC4J/OAS, Weblogic and Glassfish), but the Sun product suite includes products that compete with various Fusion Middleware 11g products (portals, ESBs, and so on). So on one hand you’ve got a “departmental” application server, which you can either licence for free by downloading the open-source version, or buy support for the fancier ‘Enterprise’ version, and on the other, an expensive, full-stack-integrated (all the way to the IDE), fully-branded strategic platform that Oracle just invested a vast amount of money into, and have been pushing like crazy onto customers the past six months. And it is the same sales team that will sell both this licensed “departmental” Glassfish. Therefore if you say the magic words like “need a cluster” or maybe “we might build a portal”, or “we are considering adopting a service-orientated architecture”, lo and behold you’ll find the molto-dinero “Fusion Middleware” based solution installed all over your sorry arse quicker than you can say “can you please explain this per-core with special CPU-architecture-loading-factor licencing schema to me once again and why is it a different price if I upgrade my hardware without adding any additional cores???“.

Let’s dissect those “key points” of Oracle’s strategy announcement:

Key Point What they meant to say
GlassFish continues as the Java EE reference implementation and as an open source project. We see it as the way to dominate the direction of Java EE for at least two years, but for Larry’s sake don’t try to use it in production.
Oracle’s strategic application server, Oracle WebLogic Server, together with GlassFish, provide world class Java EE infrastructure. Oracle’s strategic application server, Oracle WebLogic Server something something something provide world-class something something infrastructure.
GlassFish Enterprise Server and WebLogic Server expected to share core components. We are the Borg. Resistance is futile. You will be assimilated.
Oracle plans to add GlassFish Enterprise Server all WebLogic offerings. Hey, look at this cute free “reference implementation” thingy that comes free with Weblogic! You could use that to run your departmental Wiki instead of having to pay us another fortune for more Weblogic licences. Did you say “WIKI”? Did we tell you all about the great wiki-like Enterprise 2.0 features available in the Oracle Fusion Middleware 11g offering? How many test environments did you say you needed licences for?
GlassFish Web Stack maintained for existing customers. Not available for sale.
GlassFish Message Queue remains as the GlassFish messaging infrastructure. We’re not expecting to sell any licences of this. Just use Oracle Fusion Middleware’s SOA Suite 11g already. We’re fairly sure that’s got a message queue in it.
Oracle plans to license GlassFish Enterprise Server and Java System Web Server with all WebLogic Server offerings. See above.
GlassFish also available as standalone offering. Are you sure you didn’t mean to say “Weblogic”? No? Can you call back next Thursday at 2pm and ask for Fred? We’re reasonably certain he might know something about that Glassthingy.
GlassFish will continue to be supported and maintained for an extended time period for customers current on support. Well, the lawyers said we had to. We know how to do this. Ask any 10g customer.
GlassFish open source projects thrive As long as we will let them.

I know I’m a completely cynical bastard about these things, but I will wager within a few months that even if you deliberately ask for Glassfish Enterprise directly that you’ll have to fight off the Weblogic borg absolutely tooth and nail to the last man as they repeatedly try to board your IT department brandishing their integrated-wizard-driven Red Stack. I predict that, basically, after a year of not even trying to sell any Glassfish licences – because if you ask for any of the features that are in the licenced version and not the open-source one, you’ll be pushed to Weblogic (and anyway, at ten times the price they’ll prefer to sell you Weblogic as a default position, after all “Glassfish comes free with Weblogic”) – Oracle will announce, “there’s no sales in it”, then probably ditch the licenced Glassfish version completely, leaving only the open source version. Finally sometime after that they’ll cut the open source funding off and it will have to limp along without hardly any of the resources it formerly had. Maybe they’ll donate it to the ghetto of an Apache incubator project where it can die unnoticed a couple of years after that. It’s a pity because IMHO Glassfish is ten thousand times a better app server than anything Oracle ever produced, or even bought before this.

On Architecture and Craftsmanship

The science of the architect depends upon many disciplines and various apprenticeships which are carried out in other arts. His work consists in craftsmanship and technology. Craftsmanship is continued and familiar practice, which is carried out in the hands in such material as is necessary for the purpose of a design. Technology sets  forth and explains things wrought in accordance with technical skills and method.

So architects who without culture aim at manual skill cannot gain a prestige corresponding to their labours, while those who trust to theory and literature obviously chase a shadow and not reality. But those who have mastered both, like men equipped in full armour, soon acquire influence and attain their purpose.

M. Vitruvius Pollio,   De Architectura Libri Decem, 1.1.1-2. (approx 31 to 27 B.C.)

Money-is-Money 0.17

I just released a new version of Money-Is-Money, v0.17. If you’re interested, see the features here and here. My aim with it is to make the most accurate currency-aware Java money handling library available. This time I’ve added just a couple of new methods to get the whole and fractional amounts as Integers (I use throughout BigDecimal and BigInteger to represent amounts, to avoid the fatal floating point errors).

To include it into your Maven POM you’ll have to add my repository to a profile in your settings.xml:

<repositories>
  <repository>
    <id>crazy-mcphee</id>
    <url>http://modular.autonomous.org:80/artifactory-2.0.5/libs-releases-local</url>
    <snapshots>
      <enabled>false</enabled>
    </snapshots>
    <releases>
      <enabled>true</enabled>
    </releases>
  </repository>
</repositories>

At which point you can include the dependency in your pom.xml for your project:

<dependency>
  <groupId>org.autonomous</groupId>
  <artifactId>money-is-money</artifactId>
  <version>0.17</version>
</dependency>

If you want to check the source code out with SVN the release is at http://crazymcphee.net/svn/money/tags/money-is-money-0.17 and the latest trunk at http://crazymcphee.net/svn/money/trunk

Desktop lockdown

An interesting Wall Street Journal article, “Why You Can’t Use Personal Technology at the Office”, came my way courtesy of a Linked In group discussion this morning.

In terms of the article, I agree it has been my experience for many years where I have faster/better personal technology than my workplace. At one place we had to put a “business case” for getting every developer a two monitor setup. Well, it was trivial and the developers got the monitors. In my view all attempts to control the desktop are at best essentially futile, and at worst, actively counter-productive.

Nowadays I simply will refuse jobs where the “SOE” means I must use Windows. It is a question I ask at the first interview. Windows is great for people who have needs that don’t extend beyond word processing, spreadsheet, and corporate email. But that doesn’t mean those technologies have to be forced on those users! Compatible systems for all these choices exist on most major operating systems.

However I believe that technical users and developers like myself are better served with Unix or Unix-like derivatives, e.g. Linux or Mac OSX, mainly because the Unix shell is one of the most powerful standard tools any developer can learn. And I am certainly a digital rogue. I use a Macbook Pro. Well as a consultant I provide my own equipment and that’s that. I use a VMware install of Windows if I have to for some reason. Given a lot of corporate systems that we build nowadays are done as web apps, I don’t think the personal choice of desktop systems matters terribly much anyway nowadays! In my view, companies should tell the user the budget for their system and they should buy whatever they like with it. And yeah, desktop ops guys, suck that up and deal with it just like us apps guys have had to deal with “heterogeneous environments” for over a decade now. Reading the article you can see what some companies like Kraft are doing in this area and it is definitely the way of the future.

In my view the number one thing that stops desktop nirvana is the Microsoft application stack – Exchange, Outlook, Sharepoint, etc. This stuff is the spawn of Satan as far as I’m concerned – have you ever used the Exchange web app not in I.E.? It’s worse than the first versions of Squirrel Mail ! There’s not even a functional search function (at least on Exchange 2003), i.e. a total joke.

In the development context, I’m a great believer in allowing developers to choose whatever IDE they are comfortable with, that the code base ought to be totally independent of the artefacts used to edit and manipulate it. Their productivity will soar as their deep knowledge of their existing tools ceases to be a roadblock in getting stuff done. It’s really depressing when these app server / suite vendors release completely integrated environments from the production server all the way to the developer desktop. This is a productivity sink if you have to retrain your Eclipse/Netbeans/Intellij/Emacs/vi developers to retool on some new right-mouse-point-and-click-context-menu style development environment that is completely locked to the vendor. This is not a good story in my view. The vendors need to consider strong de-coupling of their deployment environments from their development tools.

Mistakes you can make with SOA

Bob Lewis has a great column this month, “What if SOA is a mistake“? His penultimate paragraph asks:

Lost in the shuffle is something basic: Programmer productivity. Friends who are hands-on with such matters tell me the available SOA development environments are less than half as productive as products like PowerBuilder and Delphi were, back when they were viable.

Putting aside the Powerbuilder and Delphi love for just one minute, this is something I’ve been banging on about now for the past year … the programming tooling that is foisted onto programmers by the choice of the deployment architecture. It’s just all wrong.

In my view, what makes a programming language really productive is notepad. Or vi, or emacs, or gvim, or textmate, take your pick. What I mean is … the programming language has to be able to be programmed with a simple editor. Yes, an advanced IDE will make things more productive, but the basics must also apply. Now a lot of SOA environments are simply not programmable without the specific IDE tied to it. Even worse, the IDEs are often completely custom jobs that require a developer to be re-trained … losing years and years of productive speed with muscle-memory style automatic ability to navigate the programmer’s usual editing tool. Seriously. This stuff is whack. A program language or an environment needs to be IDE-neutral. If you got a plumber around, would you insist that he only use the tools you supply from your home handyman kit? Or would you expect the plumber to have mastered a set of his own tools already? And making matters worse, it’s rarely programmers that choose these tools which are foisted on them. The server/deployment environment and the language used to implement need to be decoupled from the tools used to build it.

But an even worse failing of many of these SOA tool suites, is that they all strongly and irrevocably coupled to the deployment/runtime environment. Generally they totally lack the ability to keep up with modern programming practice. Like for instance, automated testing. Or even unit tests, let alone advanced and productive techniques such as Test-First approaches or Test Driven Design. The tools often lack refactoring support. All of these things are in my opinion, and in the opinion of many leading developers, absolutely essential to quality engineering practice and agile development outcomes like “delivery of working software”. Both the “delivering” and the “working” part means the whole process needs to be repeatable. That’s why automated integration testing, to name just one thing, is essential in modern development. But often the fancy custom development tooling is a complete barrier to achieving this.

But you won’t hear any of this from the big vendors. One big vendor recently announced their new version of their middleware product suite had a ‘focus on testability’, but you ask any of their presales guys to demonstrate this in an actual development environment. Ask them about continuous integration support, for example. Witness their blank looks. Their development product is completely orientated to “one button push from the IDE to production” modes of thinking the idea of continuous integration builds is almost totally antithetical to the very concepts of operation the product is organised around. They think that finally adding support for Subversion version control system, at least five years too late, is a wondrous achievement.

They are aiming for ‘programmerless programming’: of course in process just creating a new type of programmer. Every new generation of programmers simply have to learn the same hard-fought lessons of software engineering over and over again because each generation of tooling apparently scraps the paradigm over and over again in a vain attempt to create push-button, wizard-driven programming models. They nearly all suffer from ‘hello world’ programming – the simple examples that sell them to IT management are trivial to conquer using the wizards, but more complex problems (i.e. real world ones) are flat-out impossible. Thus these tools are always mirages which look great at a huge distance on the horizon but are flat lifeless salt pans of bleached skulls and bones on closer examination (or, maybe they are more like tar pits that look like a nice waterhole but one step into it and you are sucked down  to your doom).

As you might be able to tell, I am utterly contemptuous of many of these SOA tool paradigms. I have nothing against SOA itself, per se. But there is nothing more productive than a programmer who understands the importance of simple and repeatable build and deployment automation using command line tools and who knows his programming editor inside out after ten years of use. Give that programmer a better language by all means, add incremental features to that IDE, allow the programmers to continuously improve their techniques, promote professional craftsmanship, yes, yes and a thousand times yes. But no amount of drag and drop wizards, push-button deployments, and “object inspector” property editors will ever usurp that deep knowledge a good programmer brings to both his language and his personal tooling choices.

ORM-is-Dead meme

I agree with Stephan, and  Aldo; ORMs increasingly get in the way.

Collection mapping is one of those “hello world” problems. (The “hello world” example in the doco looks totally trivial and completely ideal [which is the problem], but suck-in-the-galaxy-greet-it-and-then-map-all-the-stars problem, which is more like what your real app looks like, is far less than trivial and not so easily achieved with the demonstrated toolkit). In my experience, collection mapping is the cause of all sorts of pain and performance perfidy.

Here’s a further situation where I find that they really get in the way. Consider integration with existing systems that use highly-normalised database schemas (designed by advanced SQL programmers who know what they are doing in that language).

This is not an uncommon scenario – we have this system ‘A’ which (let’s say) has all our customer shipping, product, warehousing data in it. We don’t want to replace it. But we need to extend it so that e.g. available to web systems. Or maybe it needs to be integrated into the big fancy off-the-shelf CRM we’re installing. Something like that. We need to build some sort of service layer around this system ‘A’ so that other systems can access its data and/or functions. It’s common to think that an ORM around that big complex database schema is going to help, but in my experience, it doesn’t for many of the reasons Stephan and Aldo list. That sort of highly-normal database schema in my experiences completely kills ORM object representations stone cold. You end up with so many LazyInitialisationExceptions and various other problems it sucks productivity out the developers and performance out of the system.

Also, I get a bit annoyed about ORM abstractions leaking into the web tier, this is especially prevalent with using annotations rather than ORM mapping files.

Different sorts of approaches are sometimes needed to be applied with careful thought in that scenario. Perhaps you need to model the “intermediate” domain you need directly, and then use, for example, simple DAO layers which operate directly on the domain you’ve defined. These DAO layers then might call functions (stored procs) in the database that do the problematic mapping in an efficent relational language (after all it’s the direct representation of the target data format).

I’m not saying that you have to do the above in every case, or that an ORM is always wrong, but anyway the point is, ORM isn’t a magic bullet like any other technology you have to consider carefully. Your application doesn’t always need one by default.

Out of the box experience

Every now and again we get some customers who expect that they can get a custom website, portal, or services integration done by looking at a vendor’s “out of the box” experience. This can be very frustrating for us, as we need to get into their heads that no platform will delivery any website, portal, or integration “out of the box”. I classify this as a species of magical thinking.  This sort of thinking is so persuasive among many IT systems users that they will spend $500,000 on the infrastructure and $50,000 on the development effort. They are often shocked to discover that to get all the features they demand – even when those features can be delivered trivially from the chosen platform, costs time (and therefore money) often to the equivalent value of the software licensing. The “out of the box” approach can deliver excellent results in terms of a single-point-system, let’s say a CRM (e.g. sign up for a Salesforce account) but they don’t see to integrate that CRM into their custom warehousing system (for example) and linking all of that into a comprehensive product website involves completely customised software development. Every website, portal or integration scenario is custom – always. Unless it is somehow the case that you don’t mind that your website is the default “Welcome to Apache Tomcat” page.

In recent times I’ve seen this often enough that I think it’s really a failing of the IT industry in general, and we need to educate business IT users about the various scenarios and categories of software.

The simplest analogy I can think of is to say the website or portal is the letter, and the platform is the word processor.  Regardless if you use Word, Wordperfect, Pages 09, your email program or just plain old ‘notepad.exe’, at the end of the day the time to write the letter is pretty much the same effort and therefore the task is basically identical. If you said “I want you to help me to write a letter to my member of parliament”, should  I ask you whether you’re using the new letter wizard in Word? Have you seen this great “clippy” feature? Have you considered the new upgrade to Office 2007? Tell you to buy a Mac? Or would I be better off asking who is your member of parliament and what’s the matter about? When we get these sorts of naive clients we need to concentrate their minds on what their actual problem is and the best way we can solve it, when they’ve got their head in the sand thinking about that great drag-and-drop wizard feature the vendor showed them they totally thinking about the completely wrong thing.

Agile and assembly lines

Coder Friendly’s got an interesting article about Agile Evangelism. First I’m not going to take issue with the article itself but something that he quotes from DanC’s Lost Garden on Managing Complexity:

The repetative (sic) steps that a single worker performs on an assembly line is a good example of a simple task

This is a terribly Fordist/Taylorist view to take of assembly work. That most beloved of industrial examples, Toyota, takes weeks to train its process-line workers to be able not only to do the simple repetitive steps of assembly, but also to be able to identify and rectify problems with the assembly-line. As Steven Spear, author of Chasing the Rabbit (my review here) might say, anyone who thinks that “assembly line” work is simple task probably doesn’t understand the nature of their process enough to keep up with their more innovate competitors, let alone outdistance them. Understanding your process, well more importantly, your organisational goals and how your process does or does not fulfil those goals, is critical to improving your organisation, and it leads me onto the idea of “evangelism”.

I both agree and disagree with Coder Friendly’s position:

Don’t become an evangelist yourself : look at your project and decide which methodologies is the most adapted. Sometimes, Agile is not the right one but if you see that your requirements are far from agreement or certainty, Agile approaches are probably a good choice.

I agree that “evangelism” can be politically off-putting.People don’t react well to being told their old kool-aid contains poison, and to uncritically drink this new kool-aid instead. However the main problem I have with a statement like this is that non-Agile companies use this “adaptation” argument and approach, in order to keep their poor contexts alive. This is the fallacy that’s found in ’scrum butt’: “Oh we do Scrum, but …”. The clauses after the ‘but’ are typically critical. The organisation concerned is usually trying to keep some essential organisational dysfunction alive, because of politics, resistance to change, simple misunderstanding, lack of understanding about their own process, or maybe even simple laziness. I’ve got no problem with an organisation that is mature in its use of an agile method adapting it to their local circumstance – in fact if you are doing it properly, with its emphasis on empowering teams to own their process and carrying out regular ‘inspect and adapt’ at the end of every iteration, after six or twelve months of this the process will most certainly be adapted to the local conditions.

However, the real danger is at the start of the agile adoption process, when you have people hand-waving about “can’t do that here”, for whatever value of that there is. I’ve seen it said about: pairing; story boards; acceptance criteria; automated testing; retrospectives; daily stand-up; adaptive and emergent design; and just about every other aspect of XP or Scrum process. The reason is, I think, that these organisations rarely understand what the real organisational goals actually are, and they understand even less about their existing process, to the point they just “do” the process that “is”. “It is what it is”, is a great term, and if anyone says this to you it’s just a fatalistic attempt to accept as unchanging something you can change. Accept the bad. Don’t think it could be better.  Attempts to meddle in the process, i.e. change it, induce fear, and with fear often comes timidity.

This is where you need a real agile evangelist. This is not someone who just thinks that Scrum or XP or whatever other process should be blindly substituted for the existing one, but someone who understands that process is about competitive advantage, eliminating waste, and that you have to start from a known position and continuously work towards improvement. Change is not just good, but necessary, and you need those change agents. Passion for doing things better is bad? Only in a change-averse company. So learn your existing process, thoroughly, and then learn the proposed agile process, thoroughly, and only then will you be able to ’synthesise’ a replacement. Only I think you’ll find that replacement will look a lot more like the plain old agile process than your existing unagile process. After all, if your existing process already has adequate inspect-and-adapt cycles within it (not lip-service ones, real ones), probably you are already doing and agile or lean process even if you don’t call it that, and you’re already number one in your marketplace. If not, perhaps you require the passion of an evangelist.

American Express are enablers of security threats

Peter Newton, “Director of Marketing” at American Express Australia Limited ABN 92 108 952 085, really likes Russian Mafiosa. Or maybe it’s Chinese Triads, bikie gangs, or maybe some local spivs operating out of a small rented smash repair shop near Beenleigh. Perhaps all of the above. Look at this email I got in my inbox this morning:

American Express email phishing attempt ... by American Express

American Express email phishing attempt ... by American Express. Click to see full size.

This email is signed, with a little jpg at the bottom, “Peter Newton”, Director of Marketing, because, you know, that makes it more trustworthy. BTW, at the bottom of the email (not in the picture) it says “You can contact American Express Customer Service for further information. Unfortunately we can not accept incoming emails to this address”, snail mail only, FFS.

It appears quite clear to me that American Express possesses an unbelievable commitment to training their customers to accepting any roughly legitimate-looking email or incoming phone call (see below) as authentic, and having them hand over their valuable financial information to who knows what is at the other end. This email is totally incredible – but yet it is legitimate, I checked the linked addresses before I, nonetheless, still marked it as a “phishing attempt” in Gmail. Look at those security melting links! “If you have forgotten your user ID or password, please click here” (!!!). Amex, you are not some pissy little forum I visited once three years ago that can send an email with a link to the password recovery, you are a bloody financial service. This stuff is a massive security FAIL, you idiots.

I will note in passing that my bank has a line in their online banking website that says to the effect ‘we will never email you about this service’.  They do send me an email that says “your online statements are available” but there’s not a single link in the email. Dear Amex, there is a good reason they never do that! Thank god my current credit balance is zero – although come to think of it that makes me more worried that someone could be stealing more of my credit limit right now.

As I hinted above with the comment about “incoming phone calls”, this is not the first time I’ve discovered that Amex likes to operate in complete contradiction to the rules of good security practice. A few years ago I was rung on my mobile phone by their customer service department; “This is xxx at American Express, we’d like to talk to you, but first can you identify yourself with the following information?”, ran the voice at the other end of the line. No amount of me saying otherwise could persuade them it was a very bad idea: “Well, if you are really Amex, you’ll have all that detail on file in front of you, perhaps you ought to tell me that information to prove you are who you say you are. After all you rang me … “  I said I’d call them back – the operator tried to give me the number but I cut her short and said I’d use the number I already have, thank you very much. After I cleared the issue, I asked to speak to their relevant people and attempted to explain what was wrong with the entire scenario, but I doubt it had much effect, they just claimed “procedure” and read me a lot of boilerplate about how secure they are. But they are not, and here’s the evidence.

Dolts.