by FONGO_steve » 06/22/2011
Today we experienced a failure with one of our core network elements that was service affecting for core voice and some internet service infrastructure. At approximately 11:25 am, a switch failed resulting in a loss of connectivity between 2 core sites in Toronto. Service was restored as of approximately 12:10 pm. We are investigating root cause for this failure, will be reviewing options, and implement an option which will prevent future recurrence of this issue.
In addition, our primary DNS resolver infrastructure was unreachable resulting in a higher than normal load on our secondary DNS infrastructure. We will be implementing 2 additional deployments at 2 additional sites to avoid this in the future. The consequence of the DNS load increase would have resulted in slow or lack of response from some internet / data services.
We do apologize for any inconvenience this may have caused to our customers who were trying to use their phone service during this time, and we thank you for your continued patronage.
Steve
Fongo
Development Support Specialist.