Incident - South Island connectivity - 17/03
Power reconfiguratons and replacements have been completed successfully with no customer impact.
While root cause is still being fully determined, we believe a large shunt or surge of power into the facility damaged equipment, which resulted in damage to systems and extra load being placed onto power systems at our Christchurch facility.
These power modules have now been replaced and upgraded tonight successfully without dropping load to network and server equipment inside the facility, and network services continue remain stable after initial impact. While the facility is operating at an elevated temperature, HVAC units are cooling down to normal temperatures within our tolerances as expected.
All faulting breakers and cabling have been replaced without customer impact.
Network and services remain stable. Staff remain on-site to ensure all supporting equipment remains stable before closing off.
First 63A breaker and associated cabling has been replaced without impact. Two further replacements remain and are now being worked on under observation from our team.
Network and services remain stable.
Faulting breakers, as well as some cabling, will soon be getting replaced to rectify identified issue.
While all care is being taken to continue power supply to network and server equipment, as we are dealing with electrical supply, there is the possibility that one or more feeds to racks may be lost during works and this should be taken as a hazard notification.
Our team remain on-site to plan and monitor works.
Network and services have continued to remain stable.
Electricians are onsite and have identified faulting breakers.
Plans are underway to replace faulting breakers.
Network and systems remain stable.
Network and systems continue to remain stable, however power feeding into the site continues to be unreliable.
Electrical engineers are en route to site.
Both UPS feeds and battery banks are operational which is ensuring continued service.
Fault has been isolated, and impacted customer connectivity is being restored.
Engineers are continuing to work through remaining faults.
Issue has been identified as a power issue at a core site.
Engineers are en-route, ETA 15~20 minutes.
Due to an ongoing incident at one of our Christchurch datacentre facilities, some customers may be experiencing a loss of connectivity.
This is being worked on at a top priority.
Updates to follow.
- Services / UFB Internet
- Services / Enterprise Cloud
- Services / IP Transit
- Services / Layer 2 Services