admin
07-12-2004, 09:53 AM
Let me first extend my apologies for the significant delays that were experienced during the NAC data center move on Sunday. I would also like to follow with my gratitude for your patience as we all endured this miserable experience together. Because the data center move was so difficult and certainly beyond any of our expectations, let me take a few moments to provide some additional information about what occurred.
The company that owned the previous data center was named NAC. NAC leased space, a couple of floors worth, to the provider that many of our shared and dedicated servers were placed with. It seems that relations between NAC and our provider began to be strained during their relationship. There were threats, lawsuits and Restraining orders placed between the two of them based on a soon to be expiring contract. Many unnecessary constraints were placed upon our provider including astronomical costs for remaining in their current location that far exceeded the costs they were charging any other customers for the same services. One very important point of all of this was that the contract for services with NAC was actually sold and there was a possibility that the owner of the contract could shut down the Datacenter and retain all of the equipment and your data until things were sorted out, that was the worst case, and seemed a very real possibility.
All of the above information was provided to emphatically state that a move WAS necessary, and that it was necessary in a very short time. We were notified about the move on the same day and within a few hours of when we notified our customers. We provided the same time frame and ETAs as they provided us. We didn't feel it was a good idea to hedge the times and provide longer ETAs as we felt that announcing anything longer than 6 hours was not acceptable either. The move was scheduled for a finite time and there were no exceptions or options in the matter that we were provided, or any other customers from what I have read.
The provider scheduled over 30 movers to assist the staff in boxing and moving the equipment to the data center that was approximately 10 miles away. Upon arrival at the data center, NAC would not permit the movers to enter the building and required that only the provider's employees could enter or remove items from the building. There were other delays associated with the move that were both NAC's and the providers fault. I am not fully aware of all that has happened with respect to those.
The move was not a success with respect to the timing, it took much longer than it should have. Whether this was caused by NAC's unwillingness to let them leave, technical issues or poor planning, I cannot fully determine, although I think it is probably a combination of all 3. I do want to assure you that now all of this is over, I honestly do expect smoother sailing ahead. The new datacenter is well constructed with the right protection equipment, good network connections (with more coming online shortly), and more staff to assist us and other customers with our needs. The NAC data center provided a very good track record over the last 6 months of almost 0 downtime, I look forward to an even more stable network at the new datacenter.
I will state that we had no control over what happened and wish we could have made an impact on the situation that would have improved the experience for our customers, the fact is we were forced into the situation by our providers. I will tell you what we did. In preparation for the move we put a disaster recovery plan in place whereby all servers were backed up to alternate servers in different data centers so that if an extended outage or data loss occurred, we would be able to restore a recent copy of the site information to a different location to provide for the services our customers were purchasing from us. Fortunately only one site had an issue that required access to their data due to some business requirements yesterday. Prior to the move, we shutdown all servers remotely to ensure that we wouldn't have issues with data corruption due to the servers being improperly powered off. In addition, we staffed the helpdesk with additional personnel to ensure that people were available to communicate what information we did have.
In summary I just want to say that the difficult situation we had is over. All boxes are moved and have been restored to service in a better environment than before. I would like to thank everyone again for their patience and look forward to a long future of hosting your sites and servers. And once again I will share my deepest apologies for the problems that occurred during the move. If you have any additional concerns or problems, please don't hesitate to contact me directly (dshurett@alphaomegahosting.com).
The company that owned the previous data center was named NAC. NAC leased space, a couple of floors worth, to the provider that many of our shared and dedicated servers were placed with. It seems that relations between NAC and our provider began to be strained during their relationship. There were threats, lawsuits and Restraining orders placed between the two of them based on a soon to be expiring contract. Many unnecessary constraints were placed upon our provider including astronomical costs for remaining in their current location that far exceeded the costs they were charging any other customers for the same services. One very important point of all of this was that the contract for services with NAC was actually sold and there was a possibility that the owner of the contract could shut down the Datacenter and retain all of the equipment and your data until things were sorted out, that was the worst case, and seemed a very real possibility.
All of the above information was provided to emphatically state that a move WAS necessary, and that it was necessary in a very short time. We were notified about the move on the same day and within a few hours of when we notified our customers. We provided the same time frame and ETAs as they provided us. We didn't feel it was a good idea to hedge the times and provide longer ETAs as we felt that announcing anything longer than 6 hours was not acceptable either. The move was scheduled for a finite time and there were no exceptions or options in the matter that we were provided, or any other customers from what I have read.
The provider scheduled over 30 movers to assist the staff in boxing and moving the equipment to the data center that was approximately 10 miles away. Upon arrival at the data center, NAC would not permit the movers to enter the building and required that only the provider's employees could enter or remove items from the building. There were other delays associated with the move that were both NAC's and the providers fault. I am not fully aware of all that has happened with respect to those.
The move was not a success with respect to the timing, it took much longer than it should have. Whether this was caused by NAC's unwillingness to let them leave, technical issues or poor planning, I cannot fully determine, although I think it is probably a combination of all 3. I do want to assure you that now all of this is over, I honestly do expect smoother sailing ahead. The new datacenter is well constructed with the right protection equipment, good network connections (with more coming online shortly), and more staff to assist us and other customers with our needs. The NAC data center provided a very good track record over the last 6 months of almost 0 downtime, I look forward to an even more stable network at the new datacenter.
I will state that we had no control over what happened and wish we could have made an impact on the situation that would have improved the experience for our customers, the fact is we were forced into the situation by our providers. I will tell you what we did. In preparation for the move we put a disaster recovery plan in place whereby all servers were backed up to alternate servers in different data centers so that if an extended outage or data loss occurred, we would be able to restore a recent copy of the site information to a different location to provide for the services our customers were purchasing from us. Fortunately only one site had an issue that required access to their data due to some business requirements yesterday. Prior to the move, we shutdown all servers remotely to ensure that we wouldn't have issues with data corruption due to the servers being improperly powered off. In addition, we staffed the helpdesk with additional personnel to ensure that people were available to communicate what information we did have.
In summary I just want to say that the difficult situation we had is over. All boxes are moved and have been restored to service in a better environment than before. I would like to thank everyone again for their patience and look forward to a long future of hosting your sites and servers. And once again I will share my deepest apologies for the problems that occurred during the move. If you have any additional concerns or problems, please don't hesitate to contact me directly (dshurett@alphaomegahosting.com).