Fighting Fires

I got a call about a new client.  The account manager had worked with them before when he worked for another VAR, but this would be the first time they worked with me.  Could I stop by and lend them a hand?  Well, they were in the western suburbs; stopping by was not an option.  But I called them up and got something scheduled.

Apparently, their servers had gone down.  The GoldMine and related applications had to be rebuilt and reinstalled.  They had finished that, but a few things here and there were still not right.  Their end users were getting a bit touchy.  I met up with their Director of Information Technology who led me into their server room.  I was impressed.  Most small businesses don’t have server rooms per se; generally it’s a cobbled together afterthought.  I’ve had servers stacked in coat closets, under desks and in attics.  One time, the primary server shared space with chemicals stored in large steel drums.  I didn’t stick around long enough to investigate what the odd smell in the air was.

The client’s server room was an actual room about the size of a garage.  The servers were all late model machines of the same make.  They were housed in matching system racks.  Everything was neat and clean.  I felt like I had stepped into an ad for IBM.  I got to work on getting their third party applications going again.  I fixed most of the problems, but got stuck on one because I couldn’t remember exactly what the syntax of the commands was.

“I need your user manual,” I said.  “I just need to look something up real quick.”  The IT Manager looked at me a long moment, sighed, and said he’d see what he could do.  He left me in the server room for a long time, but came back with a copy of the manual.  I took it in my hands gingerly.  The manual was warped and the pages stuck together as if they’d dried like that.  The end papers were covered in soot; it left my palms blackened.

“What happened?”  I asked.

It turned out the servers had gone down because an arsonist had set the room on fire.  There seemed to be no connection.  The guy who did it had no history with the company.  He just broke in, doused the server room with gasoline and lit it up.  They had actually had their tape backups in a fire proof safe on site.  While the safe was protection against fire, it didn’t protect against heat.  The tapes were reduced to lumps of slag.  A recovery company had been able to pull out the server hard drive platters and pull off almost all of the information (cost: $100,000, plus a new off-site data policy that needed to be implemented).

Despite the damage, I was able to find the switch I was looking for in the manual.  I quickly set up their fax service again and tested everything.  In the technical circles at my company, I have been known to put out a fire on occasion.  However I believe this was the first fire I had to help put out literally.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s