The Coffee Before the Storm

Drinking an espresso in 1842. Haven’t done this in a while. Actually, I’m kind of just staring at it, not because it tastes bad, but because I’m anticipating a terrible day, if yesterday’s events are any indication. Let me ‘splain:
Our *dreaded* customer called me up a little while ago, saying that some of their servers weren’t doing something they were supposed to (I’m not getting any more specific). I explained that they shouldn’t be doing anything, since they weren’t configured for this specific behaviour. They persisted, saying their other vendor (an authentication service) said so. Turns out this other vendor *didn’t* say so, and they plainly explained that in an email. They *did* say they wanted this behaviour, however, so the customer asked me to reconfigure the servers. This was unusual, since they like to test everything before doing … well, anything. I looked at the configuration, thinking that this could be a problem for their users, if anything went wrong, and strongly suggested testing on a backup server. I even went so far as to configure the backup server in the hopes that they would test it. But they were insistent.
In an attempt to please them, I followed their instructions, and waited for the other vendor to confirm the configuration. Late in the day, they reported not seeing any changes. So I checked on the servers last night, only to find that users were having problems logging in. I reverted the configuration immediately, and sent a note to all parties, citing the reason.
About 2 hours later the customer’s tech support team sent an emergency note, telling that their users were reporting problems authenticating. I told them what had happened, and that I fixed the problem earlier.
So, now, I’m awaiting the inevitable conference call about why this could happen, and why didn’t we do more to prevent it, etc, etc. I’ve had this pain in the pit of my stomach ever since this whole thing started, knowing fully how it would end. I feel like a psychic who has foreseen his own doom. Well, it’s not that bad, I suppose. No one’s ever proven any kind of psychic ability under controlled tests, so I’m probably overstating the matter.
At least I’ve got a mountain of correspondence to back myself up. Sure the results were strange — our servers should have performed perfectly in this situation, unless they had some kind of firewall or other network measure which interfered. But it certainly wasn’t my fault.
But I bet that at some time during the conference call, it will become my responsibility.

Comments are closed.