FUBAR – When It All Goes According to Plan


While my official title at work is “consultant”, I think of myself as a professional pessimist.  I am frequently optimistic in my off-hours.  I have my sunglasses with me every day, even when it’s raining.  My cup of Diet Pepsi tends to be half full.  I even cheer for the Chicago Cubs.  However, I’ve been working with computers too long to take anything for granted.  When I work with a client, I ask a lot of pointed questions, raise a lot of objections, reset a lot of expectations and ask for a lot of time.  While I hope for the best, I plan for the worst.

Awhile back, one of my clients wanted to take advantage of the free upgrades they pay for every year to the software developers.  Right off the bat, they were less than thrilled to find out the “free software” wouldn’t actually be free.  Their system was old and complex with lots of integration.  Half of it was compiled code with no documentation; as far as we could tell it ran on “magic”.  Since the software was considered mission critical (their business would dry up and blow away without it), I strongly recommended setting it all up in a test environment first.  That decision went all the way up to their board of directors, but it did get grudgingly approved.

The computer they ran everything on was actually a “virtual machine”; a really powerful computer running a program that acts like RAM and hard drives and computer chips.  The good news was we could copy this virtual machine pretty easily and set it up somewhere else as a test bed.  The bad news was their machine was in pretty sorry shape.  I ran into issues almost immediately.  The software threw random errors.  Even simple tasks would cause the system to lock up.  At the best of times, it’s difficult to isolate a single piece of software to check it for problems.  Even when a computer is “idle” there may be three dozen processes running in the background.  After several days (and nights) of effort, I got the system stabilized.  I ran it through the upgrade process, checked the data, checked the functionality and did more troubleshooting here and there as problems cropped up.

In the end, I developed an upgrade process that – while more complicated than sticking a CD in a drive and pressing <ENTER> – was pretty bulletproof.  I took into account all the issues their system had and while I did all of this, their employees were still using their existing system.  With the process, the client would only be down for a fraction of the time it took me to do things in the test environment.  We ended up not too far over budget.  Because of the size, age and complexity of their system, I figured I would run into some problems and had included time for that in my original estimate.

Of course, that’s not how the client saw it.  It was all my fault, of course.  I should have known what the issues were immediately.  I shouldn’t have wasted so much time trying things that turned out not to be the problem.  The mere fact I asked to test it all first showed how little I knew.   If I was truly an expert I would have been able to do it all without any practice.  I suppose a lot of people in my position would have been annoyed, but – as a professional pessimist – I believe no good deed goes unpunished.  I knew – just knew – they would be unhappy no matter how things went.  And – like I said – while I hoped for the best, I planned for the worst.

Hal_brain_room605

Leave a comment