Optus under investigation following Triple Zero failure

Optus is in the spotlight after admitting that a recent botched network upgrade resulted in the failure of over 600 Triple Zero calls in South Australia, the Northern Territory and Western Australia — with four people from affected households subsequently passing away.

In a series of statements that began on Friday afternoon and continued over the weekend, the company and its CEO, Stephen Rue, revealed that a regular firewall upgrade took place on Thursday, 18 September at around 12.30 am (Rue did not specify the time zone). Unbeknownst to Optus at the time, it transpired that established processes were apparently not followed.

As a result of the upgrade attempt, over 600 customers from the aforementioned regions — plus at least two from NSW who live near the South Australian border — found themselves unable to get through to emergency services. It appears that the upgrade created a set of circumstances where the emergency ‘camp-on’ capability, which should have routed emergency calls to other networks, failed to function in this instance.

“When the upgrades and changes were implemented, initial testing and monitoring did not indicate there were any issues with calls connecting — normal calls were connecting as they should and call volumes at a national level did not raise any red flags,” Rue said.

“[But] there was a technical failure in the system, and further, there were no alarms to alert us that some emergency calls were not making it through to emergency services.”

Rue stated that Optus only became aware of the severity of the incident when a customer contacted the company directly at around 1.30 pm that day, followed by South Australia Police at 1.50 pm.

“Once notified we stopped the upgrade, restoring Triple Zero, and began to confirm with relevant stakeholders such as police and other regulatory and government agencies and departments that we had experienced an outage impacting Triple Zero,” Rue said. This means the outage lasted for around 13 hours in length.

“Welfare checks [on affected households] commenced later that evening and into the following day, Friday (the delay due to the complexity of pulling records from the network),” Rue continued.

It was only after three fatalities had been confirmed that the company revealed the outage and its consequences to the wider public, with Rue claiming that Optus “was keen to be sure of the facts that were emerging and believed to be true” before making any statements. A fourth death was later announced, while one of the original three — an eight-week-old baby in South Australia — appears not to be connected to the outage, as the child’s grandmother was able to contact Triple Zero on a different mobile phone shortly after her own device failed.

Rue said that Optus was unaware of the connection failure for “an unacceptable gap in time” — and yet, he also admitted that at least five separate calls were made to the Optus contact centre on Thursday morning to complain that the Triple Zero service was not working.

According to Rue, “this information was not surfaced with the relevant escalation at the time” as Optus itself had not detected any Triple Zero failures. Two of the callers went on to complain to the Telecommunications Industry Ombudsman.

“This is not clearly good enough, and we are implementing a new compulsory escalation process following any customer reports of Triple Zero failures through our customer call centre,” Rue said.

Rue said the company had launched an immediate investigation into the incident, with plans to cooperate fully and transparently with all relevant government agencies and regulatory bodies and to share the facts of the incident publicly. 

“In the short term, I have put in place an immediate halt to further changes in our network system until we have a broader understanding of the events that have occurred so we can also introduce greater monitoring, testing and compliance and reviews of our change processes,” he said.

“Further, our technical teams are monitoring Triple Zero call volumes and failure rates state by state 24 hours, seven days a week.”

Speaking at a press conference on Saturday, Minister for Communications Anika Wells said she had spoken to the Chair of the Australian Communications Media Authority (ACMA), Nerida O’Loughlin, about ensuring there was a full and thorough independent investigation into what happened. “And I will be looking at what other requirements need to be applied upon providers to ensure that they alert emergency services properly, which it is clear on the facts that we know at the moment [that] Optus has not done,” she said.

Wells said she found it hard to put into words how such a failure could have been allowed to happen again, with the so-called ‘Bean Review’ having making 18 recommendations to the telco industry following the nationwide Optus outage of 2023. According to Wells, “Many of the things that happened in this outage are failures to implement some of those recommendations, including alerting the public or emergency services authorities, and that is what I find to be particularly disappointing.”

Wells said she was particularly “staggered by the way that Optus has navigated this with our governments”, having only been notified of the full extent of the outage slightly before the rest of the nation.

“We and my department, and I believe the ACMA … were first emailed a notification that there had been an outage affecting 10 calls on Thursday afternoon, about 3 pm I think from memory,” she said. “We didn’t hear anything further until 3.40 pm Friday afternoon where we were told the outage had affected about 100 calls. And then shortly after 4 pm we were told the outage had affected 600 calls. And then we found out from our department that there had been three deaths and then we were told that there would be a press conference from the CEO of Optus shortly.”

The ACMA confirmed on Monday that it had commenced its investigation into Optus’s compliance with emergency call service regulations and other related rules, which state that telecommunications carriers and carriage service providers (CSPs) must ensure that emergency calls are successfully carried to the emergency call service at all times. They must also undertake welfare checks on callers who made unsuccessful emergency calls during a significant network outage.

These are the same rules that the ACMA found Optus breached during the 2023 outage. Optus was penalised $12 million by the ACMA for these breaches.

In this new investigation the ACMA will also investigate Optus’s obligations to:

• communicate information about the outage to customers and the public, including putting relevant and up-to-date information on its website and using apps, email, SMS, other media or call centres to keep the public informed;
• notify the emergency call person (Telstra) as soon as possible about a significant network outage; and
• communicate information about the outage to other stakeholders, including the relevant ministerial portfolio department, the Telecommunications Industry Ombudsman, the National Emergency Management Agency and the ACMA.
   

The ACMA will make its findings public once the investigation is concluded.

Image credit: iStock.com/ymgerman