IT “Story Time” doesn’t sound like the most captivating publication, but I promise I won’t bore anyone by going on ad infinitum with technical drivel. I’d like to recount a very frustrating experience with a very satisfying outcome.
I was called in to a manufacturer to solve a daily server crash issue. The client complained that the server would routinely go down three times every day and, as a result, halt production, shipping/receiving, and accounting. The down time would disrupt their business to the tune of thousands of dollars per day.
In addition to having to restart their server and a bunch of other equipment each time, data integrity was suffering with the incomplete accounting entries resulting from abrupt disconnections.
I went in after hours so I could work autonomously without impeding their operation more than it was already being interrupted. I spent a considerable amount of time diagnosing every piece of hardware on the network, and despite being there for a full eight hours after 5pm, nothing went wrong!
The bane of every technician, service person and mechanic, is the elusive bug that hides the moment you start looking for it! It’s like cockroaches when you turn the lights on!
The next day, I visited to give the client the bad news that I hadn’t discovered anything wrong with their network, and then at 10:30am there was a growing crescendo of staff moans, complaints and expletives. The server had just crashed again!
I quickly took a mental snapshot of everything, hoping to see some clue of what was different… What changed? What happened?
Somewhere in my deep subconscious, I noticed that the Coffee Truck just rolled in. I know… it sounds insanely abstract, but it coincided with the server crash. Another image… the receptionist picked up the phone, pushed an intercom button and said “coffee truck” but I didn’t hear anything.
My mind immediately went to the Sherlock Holmes quote; “when you have eliminated the impossible, whatever remains, however improbable, must be the truth”.
I asked the receptionist where she was making that announcement. She responded that it was for the warehouse staff.
Could this be something? I darted out to the warehouse and asked an employee where the announcement for the coffee truck came from and he pointed high up into the ceiling at the PA speaker for the intercom system. It was a monstrous tuba of a thing, and EUREKA! …there beside it was the tell-tale jacket of a blue ethernet cable!
This wasn’t an entirely new concept to me. I had seen this before with a 600v electric truck gate motor at a distributor client, and an elevator motor in a shaft between floors of one of my financial sector clients. Ethernet does not like getting “defibrillatored” – to coin a phrase. The solution was simple… move the cable or the PA speaker. Mission accomplished!
Sometimes computer problems can point in misleading places, where you can waste a lot of time trying to fix something that’s not broken. Thinking outside the box is critical for a full diagnostic.
The moral of this story… Experience and abstract thinking is a priceless commodity in tech support!