Tuesday, January 22, 2008

Strike for the Upgrade

Beginning Thursday evening, we proceeded with consolidating documents from 6 offices into a central location in Clarksburg. That process was completed Saturday evening and we did some testing from the other 5 offices on Sunday and Monday, and while a bit slow, everything seemed to be working fine. Monday was a holiday so it wasn't a full test...today was the day when it would matter the most. Today would be the first day that the entire organization would log into the domain and attempt to work across the WAN. A true test...

That is until I received a call at 7:45 a.m. with an ominous message: The power is down in Clarksburg. Well, at least we have a never-fail system with redundancy to the Charleston office so people should still be able to get in and work. Right? Wrong. Apparently the never-fail system wasn't setup properly and didn't replicate.

So by 8:30 a.m. users were attempting to log-in and, obviously, couldn't. The Domain Controllers were down, the document system was down and there was no other system for users to log-in to. And, of course, this entire problem will be blamed on the upgrade despite the fact it is the fault of the building.

This is a major strike against the upgrade because people will now associate further progress as troublesome. Whether it is truly the fault of the new system or not, it will be assumed that everything that goes wrong is because of our move forward. It's not a good day here...not good at all. Maybe we were too ambitious for one weekend...attempting to migrate users to login to both Novell and Active Directory plus consolidate our documents in one weekend. Maybe we weren't. But one thing is for sure...there's going to be a lot more questions about this upgrade following this debacle today.

No comments: