Telstra outage caused by failure to address well-known network issues

• An independent report found that Telstra failed to prioritize known system vulnerabilities and lacked clear ownership of key network elements, contributing to a massive nationwide outage.
• The failure was triggered by a power supply replacement that reset a GPS card to the year 2006, causing a synchronization error that disrupted 45 per cent of mobile traffic and blocked hundreds of emergency calls.
• The response was significantly delayed because critical alarms were only monitored during business hours and key engineering staff were on mandatory leave at the time of the incident.

Telstra ignored warnings about system vulnerabilities months before a massive nationwide outage crippled its network. An independent review by Technology Audit Partners (TAP) revealed that the July 8 failure, which disrupted 45 per cent of all mobile calls and data sessions, was exacerbated by a lack of oversight and inadequate staffing. Most significantly, the report confirmed that hundreds of emergency Triple Zero (000) calls failed to connect during the crisis.

The investigation found the outage began at 2:50 am—nearly an hour earlier than Telstra initially disclosed. The technical collapse was triggered by a routine maintenance task: replacing a faulty power supply. This action inadvertently reset a GPS card’s date back to 2006, a synchronization error that quickly cascaded throughout the entire mobile network and rendered services useless.

Despite the severity of the situation, Telstra was ill-prepared to respond. The report noted that two key engineers were on mandatory leave when the outage occurred. Furthermore, the alarms designed to flag such errors were only monitored during business hours by a small group of employees. This lack of available expertise meant it took several hours for the company to identify the root cause of the breakdown, significantly delaying recovery efforts.

The TAP review highlighted a systemic failure in how Telstra managed its infrastructure, citing “insufficient clarity” over who was responsible for the Network Time Protocol (NTP) systems. Because the telco did not classify its timing systems as a high-risk function or a priority, it lacked visibility into the maintenance work being performed that night. This neglect left the network vulnerable to a predictable failure that internal warnings had previously flagged.

In the wake of these findings, consumer advocates are demanding urgent changes to how telecommunications companies are regulated and held accountable. The report’s revelations have fueled calls for reform, with advocates insisting that the industry must be held to higher standards to ensure that vital communication links remain functional during emergencies.

... read the full article here