Investigation begins into Sydney Trains comms outage

Sydney Trains is set to launch a full investigation into the failure of its digital radio system yesterday, which caused the network to be shut down for over an hour in the lead-up to the afternoon peak period. The failure occurred at around 2.45 pm on Wednesday and affected the entire Sydney rail network.

Sydney Trains Chief Executive Matt Longland explained that the system in question is a critical system to ensure the safe operation of the railway, controlling more than 200 base stations and the communication between trains and the rail operations centre. Without it, train crew cannot communicate with the rail operations centre and the trains therefore cannot be operated safely. The decision was therefore made to halt all trains until the issue could be resolved, with several trains brought back to platforms at low speed so that passengers could disembark. Communication to station staff was not impacted, so they were able to keep passengers updated about the outage.

Longland said the system was designed to automatically switch across to a backup in cases like this, but this switch did not occur as it should have; engineers also found themselves unable to carry out a remote reboot of the system. As the afternoon wore on, a secondary backup site was mobilised and a faulty component (a network switch) was identified in the system. Trains were able to begin operating again from about 4 pm, but significant delays and gaps in service continued throughout the evening as trains attempted to fall back into their established patterns and platforms became crowded with commuters.

There was speculation that the incident may have resulted from either a cyber attack or a software upgrade that took place over the weekend, but Longland is sceptical of these claims.

“Our early indication from our engineering reports overnight is that it wasn’t cybersecurity-related, this incident,” he said. “It was a failed component in one of our IT systems. That component has been replaced, and it’s operating reliably at the moment. We’re obviously monitoring it very closely.

“We will undertake a full investigation over the next couple of days, including going through all of the logs of the systems to understand what caused this failure, and the key is about understanding how we can ensure it doesn’t happen again.”

Longland said the radio system, supplied by Austrian company Frequentis, has been in place for several years and was operating reliably up until yesterday.

“This kind of incident hasn’t occurred in the past,” he said. “The system has been installed since 2016, so the system is reliable; we haven’t had any issues like this. We’ll continue to monitor it; we’ll investigate the root cause to make sure that this sort of thing doesn’t happen again.”

Image credit: iStock.com/imamember