Overview

OSB installation in a development environment consists of a completely separate Weblogic instance and yet another ‘special installation’ of Eclipse. You can’t use existing Eclipse installations. Nor is it recommend to use one of the five other existing Weblogic instances that Oracle products thoughtfully demand their own copy thereof (SOA Suite, I am looking at you).

Installation process

This is the procedure for installing OSB development environment.

As network firewall policy prevents the downloading of files, you will need to use installation media located within <companyname> network. Take the files conveniently stored on network server <location redacted> and copy them into a directory on your local computer. This directory is hereafter referred to as ‘installation files directory’ below.

Install Weblogic

1. Go to the installation files directory.
2. Run the executable file wls1033_oepe111150_win32.exe
3. This action launches the Oracle Installer for the Weblogic instance. It takes some time to run so contemplate the meaning of your life for a few minutes while it does. Maybe get a cup of tea. Stretch your legs. Think about lunch or your next holiday. File a support request for more memory. You should get at least 8GB.
4. Okay that’s it, you’ll finally see the Oracle Weblogic 10.3.3.0 installer. Think about the fact that Oracle sees fit to put “instructions” on the first screen telling you what the “Next” button does. This is advanced JEE server technology you’re installing, and potentially, Oracle think you don’t know what the “Next” button does. Maybe this reflects the level of experience inside Oracle, or perhaps it is indicative of the depth of respect in which Oracle holds their customers.
5. Press the “Next” button. The next screen you will see  is very important. If you fsck it up you’ll have to uninstall and start over. It does not have any instructions, but that’s OK, we’ve done it here in this wiki page for you. Pay attention.
6. DO. NOT. ACCEPT. THE. DEFAULT. Especially if you installed SOA Suite or some other Oracle product before you installed this one. Double especially if you still want that product to work.
7. SELECT “Create a new Middleware Home” (THIS IS VERY IMPORTANT)
8. TYPE a name of an Appropriate Directory. I used “C:\Oracle\OSB_Middleware”
9. PRESS The “Next” button. Use the “Back” button if you need to see the instructions about how to use the “Next” button that Oracle conveniently provided for you on the first Screen. No, don’t do that. If you do that you’ll probably have to do this proceedure again and it will just make this entire experience last longer than it needs to. This is not a recommended practice.
10. The next screen demands that you give Oracle your Email Address to get “security updates”. It also wants a thing called your “My Oracle Support Password” (I suspect this is what might have been known as Oracle TechNet). As we both know, the best possible security measure is not to give out your password to strange programs that demand it.
11. As I planned on giving them my <workcompany> email, and as my Technet sub doesn’t use that Email address, I also unchecked the “I wish to receive security updated via My Oracle Support” check box.
12. Are you Sure? YES I’M VERY SURE. I would like to be “ignorant of security updates” and also Oracle spam. Ignorance is Bliss.
13. Look at that, I can’t give them my email address after all, what they really meant was “type your Oracle Support user id”. Press “Next”.
14. Now you have to choose whether you want a “Typical” installation or a “Custom” one. I chose “Typical”, which, being an Oracle installation, I expect to require an 8-core 64GB RAM 2TB SAN SSD -based supercomputer with a external 4-way Oracle RAC for configuration (“infrastructure”) DB in order to have enough grunt to service about 3 requests a minute. Press “Next”.
15. There’s a lot of choices here about the various subdirectories under the Middleware Home Directory that you created further back. I recommend accepting the defaults, but you can probably cause yourself countless of fun trolling on the My Oracle Support forums as you get ever-more-desparate for a solution to a very obscure problem that the phone support have no idea about and that was likely caused by you mucking about these defaults, causing the support personel to simply recommend you to reinstall the product, if you really feel the need to change them here.
16. If you can remember the instruction about the use of the “Next” button at step 4, then Press “Next”.
17. Now you can choose whether you want to put the shortcuts for “All users” or just you (“Local user”). If you are the BOFH I recommend “All Users”. As this is the default, we can all safely assume that the BOFH works for Oracle and is now responsible for designing their installation processes. Accept the default, and press “Next”.
18. The next screen is a summary of what you’ll be installing. You can also select each item and see a summary of what it does. Ponder the mystery of Oracle, and press “Next”.
19. Keep pondering that mystery while Oracle Weblogic Server 11g Release 1 (10.3.3.0) is installed. It takes a little bit of time. While it does that, you might to book that Holiday, get another cup of tea, or chase up that support request for the additional RAM you’ll be soon needing.
20. Congratulations! Installation is complete.
21. I opted to leave the “Run Quickstart” option checked.
22. Press “Done”. There are no onscreen instructions for this button.
23. Quickstart will run. It’s just a link farm.

Install Oracle Enterprise Pack for Eclipse (OEPE aka “Special Eclipse”)

1. Go to the installation files directory
2. Extract the file oepe-galileo-all-in-one-11.1.1.5.0.201003170852-win32.zip … I used 7zip and made sure to put it into a subdirectory of the installation files directory.
3. It’s kind of pretty big, takes a cople of minutes.
4. Just like a regular version of Eclipse, once this is unzipped, it’s installed. However it’s not a regular version of Eclipse. It is a “special” Eclipse that went to “special” school.
5. Although this directory can probably live anywhere, it’s a good idea to copy this directory into the new Oracle Middleware Home that you created when you installed Weblogic. Look I really have no idea why this is the case, however, it’s not good to anger the Oracle by using you regular development directories. ORACLE_HOME sweet ORACLE_HOME it is then.

Install OSB and OSB Dev Tools

1. Go to the installation files directory.
2. Extract the file ofm_osb_generic_11.1.1.3.0_disk1_1of1.zip … again I used 7zip and made sure to put it into a directory underneath the installation files directory.
3. Enter this directory. Enter the directory “osb” that will be created underneath it. Note that even though the file said “disk1_1of1″ in the file name that underneath here there’s two directories, Disk1 and Disk2.
4. Go into the directory “Disk1″
5. Run the executable file “setup.exe”
6. A DOS window opens which asks you for the location of a JRE in order to use Oracle Universal Installer. Probably. Exactly why the Weblogic installer didn’t need to know where the JDK was, I do not know. Probably it used a sensible installer rather then the Oracle Universal Installer. I don’t ever think I’ve ever seen an machine with two Oracle installations on it that didn’t also have two or more installations of the Oracle Universal Installer also installed on it. Its name perhaps means that it installs itself universally, rather than it is a product which has a universal use for installing other software. Ponder the mystery of the Oracle.
7. A JRE will be located in the original Oracle Middleware Home that you created when you installed Weblogic. In fact there’s at least two (Sun JDK and JRockit). Use the Sun JDK. For example, my value for the JDK was “C:\Oracle\OSB_Middleware\jdk160_18″. Press Enter.
8. Now the Universal Installer will actually attempt to install something. It says “You are about to install the Oracle Service Bus (OSB) and may install the Oracle Service Bus IDE and Oracle Service Bus Examples (OSBE). Before proceeding, make sure that you have installed and configured Oracle WebLogic Server 11g. If you want to design OSB applications in Eclipse, make sure Oracle Enterprise Package for Eclipse (OEPE) is installed.” Which is all true if you’ve been following this guide.
9. Press “Next”. There’s no instructions in this program for the use of the “Next” button. Someone ought to file a change request for that.
10. Now you can choose whether you want a “Typical” installation or a “Custom” one. Typically, choose “Typical”.
11. Press “Next”. I tried looking for online help here about the use of the “Next” button but I did not find anything.
12. It does a prerequisite check. It should pass, and if it doesn’t, you are probably screwed. If it does, you will be able to Press “Next”
13. At this next screen DO NOT ACCEPT THE DEFAULTS.
14. Choose the Oracle Middleware Home that you installed the Weblogic into at the first part of this installation procedure. E.g. I chose “C:\Oracle\OSB_Middleware”.
15. Once you do the previous step, ff you followed the instructions for the Special Eclipse (OEPE), it will have found it automatically. If not, choose the location where you installed the Special Eclipse (OEPE Location). For example, my value was “C:\Oracle\OSB_Middleware\oepe-galileo-all-in-one-11.1.1.5.0.201003170852-win32″
16. Press “Next”. Did you know that the Oracle at Delphi was a priestess called the ‘Pythoness’ who answered your question using gibberish verse. A Male Attendant of the Pythoness interpreted her raving mad gibberings and told you what they meant. For a fee. Does this sound familiar?
17. Review the installation details. When you are sure they are correct, press “Install”.
18. OSB will now install. It takes a little time so run those errands, go to lunch, get a coffee, dream of the Holiday you just Booked. I’d tell you to install the new 8GB of memory that you ordered before which surely has arrived by now, but that would mean turning your computer off. Best to wait until it’s finished then.

The problem for Glassfish, as the second sentence of this ServerSide article states (see it straight from Oracle’s mouth here, and see also here) is that Oracle view it as being used for “non-mission critical department apps”. Glassfish’s superior technology (or otherwise) just doesn’t come into it. It’s not a factor (as it rarely every is).

Not so long ago Oracle spent a big wad of money acquiring an app server (Weblogic) and then a stack of more money porting all its other products into it and branding the resulting mess platform “Oracle Fusion Middleware 11g”. Now not only do they have their third app server (OC4J/OAS, Weblogic and Glassfish), but the Sun product suite includes products that compete with various Fusion Middleware 11g products (portals, ESBs, and so on). So on one hand you’ve got a “departmental” application server, which you can either licence for free by downloading the open-source version, or buy support for the fancier ‘Enterprise’ version, and on the other, an expensive, full-stack-integrated (all the way to the IDE), fully-branded strategic platform that Oracle just invested a vast amount of money into, and have been pushing like crazy onto customers the past six months. And it is the same sales team that will sell both this licensed “departmental” Glassfish. Therefore if you say the magic words like “need a cluster” or maybe “we might build a portal”, or “we are considering adopting a service-orientated architecture”, lo and behold you’ll find the molto-dinero “Fusion Middleware” based solution installed all over your sorry arse quicker than you can say “can you please explain this per-core with special CPU-architecture-loading-factor licencing schema to me once again and why is it a different price if I upgrade my hardware without adding any additional cores???“.

Let’s dissect those “key points” of Oracle’s strategy announcement:

Key Point	What they meant to say
GlassFish continues as the Java EE reference implementation and as an open source project.	We see it as the way to dominate the direction of Java EE for at least two years, but for Larry’s sake don’t try to use it in production.
Oracle’s strategic application server, Oracle WebLogic Server, together with GlassFish, provide world class Java EE infrastructure.	Oracle’s strategic application server, Oracle WebLogic Server something something something provide world-class something something infrastructure.
GlassFish Enterprise Server and WebLogic Server expected to share core components.	We are the Borg. Resistance is futile. You will be assimilated.
Oracle plans to add GlassFish Enterprise Server all WebLogic offerings.	Hey, look at this cute free “reference implementation” thingy that comes free with Weblogic! You could use that to run your departmental Wiki instead of having to pay us another fortune for more Weblogic licences. Did you say “WIKI”? Did we tell you all about the great wiki-like Enterprise 2.0 features available in the Oracle Fusion Middleware 11g offering? How many test environments did you say you needed licences for?
GlassFish Web Stack maintained for existing customers.	Not available for sale.
GlassFish Message Queue remains as the GlassFish messaging infrastructure.	We’re not expecting to sell any licences of this. Just use Oracle Fusion Middleware’s SOA Suite 11g already. We’re fairly sure that’s got a message queue in it.
Oracle plans to license GlassFish Enterprise Server and Java System Web Server with all WebLogic Server offerings.	See above.
GlassFish also available as standalone offering.	Are you sure you didn’t mean to say “Weblogic”? No? Can you call back next Thursday at 2pm and ask for Fred? We’re reasonably certain he might know something about that Glassthingy.
GlassFish will continue to be supported and maintained for an extended time period for customers current on support.	Well, the lawyers said we had to. We know how to do this. Ask any 10g customer.
GlassFish open source projects thrive	As long as we will let them.

I know I’m a completely cynical bastard about these things, but I will wager within a few months that even if you deliberately ask for Glassfish Enterprise directly that you’ll have to fight off the Weblogic borg absolutely tooth and nail to the last man as they repeatedly try to board your IT department brandishing their integrated-wizard-driven Red Stack. I predict that, basically, after a year of not even trying to sell any Glassfish licences – because if you ask for any of the features that are in the licenced version and not the open-source one, you’ll be pushed to Weblogic (and anyway, at ten times the price they’ll prefer to sell you Weblogic as a default position, after all “Glassfish comes free with Weblogic”) – Oracle will announce, “there’s no sales in it”, then probably ditch the licenced Glassfish version completely, leaving only the open source version. Finally sometime after that they’ll cut the open source funding off and it will have to limp along without hardly any of the resources it formerly had. Maybe they’ll donate it to the ghetto of an Apache incubator project where it can die unnoticed a couple of years after that. It’s a pity because IMHO Glassfish is ten thousand times a better app server than anything Oracle ever produced, or even bought before this.

Lost in the shuffle is something basic: Programmer productivity. Friends who are hands-on with such matters tell me the available SOA development environments are less than half as productive as products like PowerBuilder and Delphi were, back when they were viable.

Putting aside the Powerbuilder and Delphi love for just one minute, this is something I’ve been banging on about now for the past year … the programming tooling that is foisted onto programmers by the choice of the deployment architecture. It’s just all wrong.

In my view, what makes a programming language really productive is notepad. Or vi, or emacs, or gvim, or textmate, take your pick. What I mean is … the programming language has to be able to be programmed with a simple editor. Yes, an advanced IDE will make things more productive, but the basics must also apply. Now a lot of SOA environments are simply not programmable without the specific IDE tied to it. Even worse, the IDEs are often completely custom jobs that require a developer to be re-trained … losing years and years of productive speed with muscle-memory style automatic ability to navigate the programmer’s usual editing tool. Seriously. This stuff is whack. A program language or an environment needs to be IDE-neutral. If you got a plumber around, would you insist that he only use the tools you supply from your home handyman kit? Or would you expect the plumber to have mastered a set of his own tools already? And making matters worse, it’s rarely programmers that choose these tools which are foisted on them. The server/deployment environment and the language used to implement need to be decoupled from the tools used to build it.

But an even worse failing of many of these SOA tool suites, is that they all strongly and irrevocably coupled to the deployment/runtime environment. Generally they totally lack the ability to keep up with modern programming practice. Like for instance, automated testing. Or even unit tests, let alone advanced and productive techniques such as Test-First approaches or Test Driven Design. The tools often lack refactoring support. All of these things are in my opinion, and in the opinion of many leading developers, absolutely essential to quality engineering practice and agile development outcomes like “delivery of working software”. Both the “delivering” and the “working” part means the whole process needs to be repeatable. That’s why automated integration testing, to name just one thing, is essential in modern development. But often the fancy custom development tooling is a complete barrier to achieving this.

But you won’t hear any of this from the big vendors. One big vendor recently announced their new version of their middleware product suite had a ‘focus on testability’, but you ask any of their presales guys to demonstrate this in an actual development environment. Ask them about continuous integration support, for example. Witness their blank looks. Their development product is completely orientated to “one button push from the IDE to production” modes of thinking the idea of continuous integration builds is almost totally antithetical to the very concepts of operation the product is organised around. They think that finally adding support for Subversion version control system, at least five years too late, is a wondrous achievement.

They are aiming for ‘programmerless programming’: of course in process just creating a new type of programmer. Every new generation of programmers simply have to learn the same hard-fought lessons of software engineering over and over again because each generation of tooling apparently scraps the paradigm over and over again in a vain attempt to create push-button, wizard-driven programming models. They nearly all suffer from ‘hello world’ programming – the simple examples that sell them to IT management are trivial to conquer using the wizards, but more complex problems (i.e. real world ones) are flat-out impossible. Thus these tools are always mirages which look great at a huge distance on the horizon but are flat lifeless salt pans of bleached skulls and bones on closer examination (or, maybe they are more like tar pits that look like a nice waterhole but one step into it and you are sucked down  to your doom).

As you might be able to tell, I am utterly contemptuous of many of these SOA tool paradigms. I have nothing against SOA itself, per se. But there is nothing more productive than a programmer who understands the importance of simple and repeatable build and deployment automation using command line tools and who knows his programming editor inside out after ten years of use. Give that programmer a better language by all means, add incremental features to that IDE, allow the programmers to continuously improve their techniques, promote professional craftsmanship, yes, yes and a thousand times yes. But no amount of drag and drop wizards, push-button deployments, and “object inspector” property editors will ever usurp that deep knowledge a good programmer brings to both his language and his personal tooling choices.

American Express email phishing attempt ... by American Express. Click to see full size.

This email is signed, with a little jpg at the bottom, “Peter Newton”, Director of Marketing, because, you know, that makes it more trustworthy. BTW, at the bottom of the email (not in the picture) it says “You can contact American Express Customer Service for further information. Unfortunately we can not accept incoming emails to this address”, snail mail only, FFS.

It appears quite clear to me that American Express possesses an unbelievable commitment to training their customers to accepting any roughly legitimate-looking email or incoming phone call (see below) as authentic, and having them hand over their valuable financial information to who knows what is at the other end. This email is totally incredible – but yet it is legitimate, I checked the linked addresses before I, nonetheless, still marked it as a “phishing attempt” in Gmail. Look at those security melting links! “If you have forgotten your user ID or password, please click here” (!!!). Amex, you are not some pissy little forum I visited once three years ago that can send an email with a link to the password recovery, you are a bloody financial service. This stuff is a massive security FAIL, you idiots.

I will note in passing that my bank has a line in their online banking website that says to the effect ‘we will never email you about this service’.  They do send me an email that says “your online statements are available” but there’s not a single link in the email. Dear Amex, there is a good reason they never do that! Thank god my current credit balance is zero – although come to think of it that makes me more worried that someone could be stealing more of my credit limit right now.

As I hinted above with the comment about “incoming phone calls”, this is not the first time I’ve discovered that Amex likes to operate in complete contradiction to the rules of good security practice. A few years ago I was rung on my mobile phone by their customer service department; “This is xxx at American Express, we’d like to talk to you, but first can you identify yourself with the following information?”, ran the voice at the other end of the line. No amount of me saying otherwise could persuade them it was a very bad idea: “Well, if you are really Amex, you’ll have all that detail on file in front of you, perhaps you ought to tell me that information to prove you are who you say you are. After all you rang me … “  I said I’d call them back – the operator tried to give me the number but I cut her short and said I’d use the number I already have, thank you very much. After I cleared the issue, I asked to speak to their relevant people and attempted to explain what was wrong with the entire scenario, but I doubt it had much effect, they just claimed “procedure” and read me a lot of boilerplate about how secure they are. But they are not, and here’s the evidence.

Dolts.

As for non-Google email, my work uses Exchange – and I can’t reach an secure port for that without firing up the work VPN, so usually I use the shockingly clunky webmail alternative anyway – so its not like I am losing out on any functionality. I have a .mac address which truth be told doesn’t get used that much – and Gmail can suck in pop-based email into it’s own accounts anyway. It can be used with an unlimited number of Gmail accounts.

If you are a Gmail user on a Mac, I’d definitely recommend having a look at Mailplane.  It’s a nice way to get the Gmail-native way of handling email (which generally I love)  onto the Mac. I think its needs some improvement, such as integration into Apple’s iCal, and perhaps, if possible, the abstraction of some of the Gmail interface elements (such as gmail’s list of labels) out into native interface widgets.

Thunderbird 2 doesn’t integrate with the Macintosh Address Book, so that rules it out straight away, had to go straight to Thunderbird 3 beta. I got the latest, beta 3, released in July.

After about 5 minutes with Thunderbird 3 beta I was telling it to go fsck itself. Here’s why:

1. Installing Thunderbird 3 – what’s with the “@googlemail.com’ for Gmail? I use ‘@gmail.com’ and always have and always will. That’s the option I want. Oh sweet lord, it will force me to use @googlemail.com – that’s a disaster right there. Can be edited later, however, I discover. But still.
2. Thunderbird 3  “integrates” with the Mac address book, which is to say it will pull addresses from it, but it does this through it’s own horrible, horrible, interface. Bad Mozilla! I want to use – at all times – the Apple ‘Calendar.app’ and ‘Address Book.app’, OK? Their user interfaces. Not yours.
3. I got some mail with pictures in it. I can view the pictures by selecting “view external content” button for this one email. If I select the “always view external content for xxxx” it wants to add the sender to the address book (it’s internal address book). No Mozilla, that’s not what I’m asking for. I just want the graphics for that newsletter I get sent to be shown without me pressing extra buttons every time I get this newsletter. I don’t want their address in my address book!
4. Arrrrgh! My eyes! Jesus it’s ugly. And clunky. It feels like Windows all over again. Good lord. I can’t find any way to change the list display fonts. It’s probably in there somewhere, I will admit, but I can’t find it.
5. Every single folder, I have to individually select “view by thread”, as far as I can tell.
6. Search is an entirely separate window/dialogue. I can’t search more than one account at once. “Search Subfolders” at the top level includes Spam folder. There doesn’t seem to be a way to add a criteria that “Message is not spam”.
7. The terrible search function is the only basis for “Smart Folders” that I can find so far, therefore it won’t work for what I need. In Mail.app, I use three smart folders:
   1. All unread mail in any account or sub-folder thereof
   2. All mail in the Inbox of any of my three “personal” accounts, that is, mail directly addressed to me personally.
   3. All mail in the Inbox of my work mail account.
8. On the plus side, it does have good support for keyboard commands. I’m an old mutt user (before that, elm) so I love keyboard commands in my mail program. Mail.app has rubbish support for this.
9. Also on the plus side, Thunderbird’s threading display is a little clearer. Mail.app just puts all messages in a thread under the one hierarchy, you don’t get any sense of what message is a reply to which.

OK  Apple Mail.app lost some sent mail  once in the middle of a godawful crash (but a crash nonetheless), and I’ve always hated its horrible keyboard support. However maybe I am willing to take those risks and annoyances if this is the alternative experience. I know that Mozilla is open source, and full of all that open sorcery goodness, but I live and die via my email program, right after my IDE, and I want a pretty nice experience with it. Thus, I’m now in a bind – use a program that just proved to me its data integrity is not 100%, or use a program that that just gives me a truly horrible user experience and wants to screw with my choice of Calendar and Address Book?

Mac email program suggestions in the comments if you like (and no, don’t suggest Entourage, it’s completely off the menu).

UPDATE: I should mention that, for me, the deal killers are points 2, 6, and 7.

But the real kicker was a presentation on Agile Integration, by one of the vendor’s partners (and in the interests of disclosure, a competitor to my own employer). A few slides in and there’s the inevitable “what is agile” slide with a standard dictionary definition and some lip-service to previous history. Now the thing here of course, is that the word Agile in software development parlance is by now a well-entrenched piece of jargon – it has a specific meaning to most people that is fairly precise and different to just the garden-variety common English usage. This is the purpose of jargon in a discipline: to be precise in communication. And the agile manifesto, nearly 10 years old, just means in effect, if you’re in the software development game you just can’t redefine the term to mean something else.

Except of course, you can, if you do, and you’re big enough and enough people listen to your incredibly mangled marketoid-speak. But implementing a SOA solution is not Agile, no matter how many times you make an incredibly weak case that it is or how many times you repeat any other platform-orientated mantra. Yes, you read it right, according to the vendor’s partner at least, SOA is Agile.

This banality made me so angry, it was very lucky that straight after there was the morning coffee break because I was seething and it took all my self-discipline not to launch into an attack during the questions (easy picking, because there were no other questions asked!). I had to vent at my poor colleagues in the break.

Anyway, on my admittedly pessimistic reading it means that Agile Is Dead, as pretty soon we are going to find clients telling people, “Yeah we do agile, we bought the million-dollar SOA package off that vendor”, and then they’ll be saying the reason their integration projects are still failing is because they haven’t installed the very latest patchset, or upgraded to all-new LEANsuite 17.4.15.8 release B with the optional drag-n-drop Super-Agility add-in for Portals, Release 16f (the “f” is for “fail”).

At sushi train lunch afterwards, we made the observation that pretty much the vendor wants to be all the food groups all at once. Kind of like this bit of gluggy sushi I mistakenly lifted off the train:

All the food groups. Nasty.

Now I’m not saying that you can’t do “SOA” with an agile process, if that’s your preferred architectural style, then if you are committed you can certainly develop it in an Agile fashion. You can’t afford however, to confuse the two things. If you do, that picture above is  what your enterprise will look like too if you mistake the buzzword bingo offered by vendors for actual insight into agile development process. Talk to an expert instead.

Sorry to be so negative, the whole morning pretty much sapped my will to live.

First, I no where made any “assertion that if you are putting ‘//TODO’ markers in your code, you’re somehow not properly following the Scrum process” – I said if you find yourself using ‘//TODO’ markers in your code you are being prevented from getting to done-done and that you should “Identify the problem, propose a counter-measure and fix the process“. I can’t emphasise that enough. “Fix the process” does not mean “follow scrum” but rather to use your team’s collective brain to work out what stops you from fixing the code then and there and propose a solution to the problem.

I think ‘//TODO’ should either be in the current story (I will concede that you might want to use a ‘//TODO’ to remind yourself about tasks in your current story, i.e. “come back here tomorrow when I add the validation”) or in the “backlog”. I put “backlog” in quotes there to point out I mean “backlog” in the broadest sense, if you using Scrum or XP that’s actually where the work-to-be-done is physically kept, but if you have some other process then your “backlog” is kept somewhere else, and what is preventing you from adding to it?

I will ask a question of the anonymous poster – at what point exactly  is a ‘//TODO’ comment left decaying in the codebase without a story for days or weeks a good idea? Or is it that you just do not want to make what it is being done in the code base transparent to those who are paying your bills?

I will contend; ‘//TODO’ never gets done because the product owners never see them. They act as an opaque barrier to open communication. If you find a ‘//TODO’, lets say a big major refactoring will be needed to support a story you already see in the backlog, what’s the problem with having an open and honest communication with the team about the story’s estimate? If it’s a future story, i.e. feature related, it should be in the “backlog”; if it is code or design related which impacts you now, either it should be done then and there (refactor mercilessly) or it’s a bug which you either fix or raise a bug report for, and if it’s just some nebulous “improvement” either do it now or YAGNI. If you’ve got great tooling like Mylyn for Eclipse you can drop a “task” at the code point and move on. The point there is the “TODO” isn’t obscured away in the code but rather visible to all because the ‘//TODO’ is now either in your Wiki, your task-tracking tooling via it’s Mylyn integration, or is included as part of a story card on the story board.

If you can’t refactor mercilessly – what the post is really about – all the usual excuses apply, i.e. “we’re not confident about the code base not breaking in unknown ways” (i.e. no tests), “it has to be done by tomorrow,” et cetera, and all of these are symptoms of dysfunction in your organisation.

People love their little comfort zones and they don’t like being pushed out of them, that’s for sure, programmers included (yeah, and that’s a me too). Habit is not an excuse for avoiding improvement.

Here’s a great imaginary conversation from the C2 wiki entry about YAGNI that I think applies here:

“But Ron, I know how to do it right now, and later I might not.”

“So, Sam, you’re telling me that this class you’re writing is so complex that even you won’t be able to maintain it?”

I’m working currently for a client, building a portal which will serve tens of thousands of users. In order to deliver reasonable functionality, we have to integrate to about a half-dozen other critical systems. Email servers (two different types/styles). Calendars (three different sources of calendar information). ERP systems containing customised enterprise information, HR and payroll, content management, identity management, and a couple of specialised web apps. This is supposedly held together with a single-sign-on system. The Portal, SSO, and ERP system all come from the one vendor, I won’t say which one, but it’s a big, major vendor. One Throat To Choke, they say. Well I call bulldust.

The thing is, the only throat’s that’s got any damage on it is the poor client’s, who has managed to have it’s own throat cut wide open and lies bleeding money on the pavement.

The really big gotcha, the huge hole in the way things get pitched to clients from vendors with the “we’ll take the blame” line, is the downright lies that get told to the client about the ease of integration. The portal technology we have, simply cannot propagate identity information across to the ERP’s web services (it’s even more fundamental than that, really). The whole part of this shooting match all comes from the same vendor – portal, ERP and authentication mechanisms. There is a slide in existence in a powerpoint that I’ve seen which claims this integration is trivial. I’ve seen a document from the vendor that claims this is possible, and gives a detailed procedure. But when you try it, it doesn’t work.

Worse, after you struggle with this integration for several days, and log a support call to the vendor (who by now, having banked the cheque, has disappeared from the site completely). Well, that’s a whole new kettle of fish trying to tell each and every newly minted support technician who deal with your support requests just what is meant by WSRP (you can substitute your own appropriate acronym which is central to your particular technology if you like), or even, the logical sequence of what you’re trying to do, which things are of significance, the repeated asking for the same basic irrelevant information over and over, the failure to understand what the question is, ad nauseum. Oh, then and only then do the special “support notes” come out, the ones which which warn against the particular configuration of products that the client has been sold, when the sales slides, and available documentation, and the sales pitch, told exactly the opposite as to what you get after you actually try it and experience the massive whoosh of fail that gushes out the products concerned. I’m just lucky that mostly, it’s not me who is having to deal with this particular problem. I grieve for my colleague though. We’ve lost days and days on what is a completely critical integration. Sold on a lie by people who don’t even know what it is they are selling. Where’s the blame, where’s the throat being choked? The vendor won’t take reponsibility for their own damn product suite’s lack of interoperability!

Just the other day on a different and unrelated support call for a different problem I had to explain how to read a stack-trace, twice, to the support personel involved. Although we cleared the support problem ourselves, that is we found a tedious work-around that at least got us back on the path of writing code, we want an explanation as to what the product is actually doing, so we’ve left the job open. But in that case, the support feedback has been dead a week with no follow up on the matter with the job still critical, unresolved and outstanding, as far as the vendor should be concerned (because we told them so).  Hello? <cue cavenous reverb effect>. <tumbleweeds>.

The hilarious thing is, the funky little systems the client has already installed, with competent programmers in the client’s own employ looking after the other end, look like a joy to integrate against: trivial, almost. Developer on site.

I can’t believe in this day and age these big vendors sell anything to anyone. Hasn’t everyone been burnt like this one too many times? I’ve dealt with other similar big vendors in the past. They are all the same basically. Snake oil salemen selling an oily tar-pit of overcomplexification as if it were the acqua vitae.

anon 9:04PM
[about that code]

crazymcphee 9:05 PM
well, it WAS perfect … CRAZY perfect

anon 9:06 PM
lol… no doubt it even spans a 2yr time frame and 2 organisational puchases later, with no impact

crazymcphee 9:07 PM
exactly. all future and past and alternative universe combinations taken care of forever. no changes necessary.

anon 9:08 PM
sweeet… it should be a product

crazymcphee 9:08 PM
oh it already is just not in this instantiation of the multiverse

anon 9:08 PM
this instance has finished run level 3 yet

crazymcphee 9:08 PM
but in those OTHER multiverses, Oracle already bought it for $500mil

anon 9:09 PM
hahaha, i can see you partying with ellison and his geisha girls ;D

crazymcphee 9:09 PM
tried a ‘sudo shutdown’ but something’s threadlocked the kernel

anon 9:10 PM
then I stepped in a core dump

crazymcphee 9:10 PM
that’s why i’m hard at work building a mutiverse portal so i get me a slice of sweet ellison geisha-girl action

and here you are thinking about some OTHER sort of portal when i said i was working on an ‘Oracle 10g Portal Implementation’

anon 9:11 PM
don’t forget to add some proprietary and intrusive components that spread like a virus and grind all the other appservers to dust

crazymcphee 9:12 PM
well, that’s what at least half of those 10^100 multiverses full of new Indian IT grads are working on

I just send them a 500 page spec each month and they will get it to me at sometime before the death of this multiverse

anon 9:13 PM
that means it should be about ready by now, well as in it probably compiles in at least one of those multiverses, will be fully cmm lvl 5, yet not actually do what you asked

crazymcphee 9:13 PM
(oh by ‘spec’ i mean, a drunken rant shouted into my mobile phone on the walk home)

well, yes, but i’m fully expecting it will meet oracle’s stringent marketing requirements

anon 9:14 PM
well shit, they have to do some work, what do they expect, 4 u to write the code as well

yes marketing tickbox on the packaging is the only requirements they really need

crazymcphee 9:15 PM
yeah, i mean, hell, we pay at least $5.50 a day per developer …

oh we don’t tell the DEVELOPERS about the marketing tickbox requirements! they are super-secret. we just slap those on the box at the end. when i say ‘end’ i mean end of the box design process which has been ready for about 8 months now.