When time goes wrong: what the Telstra outage teaches us about accuracy

When time goes wrong: what the Telstra outage teaches us about accuracy

Md mahdi 1q3fY9dyB9o unsplashThe recent Telstra nationwide outage caused havoc across Australia, with widespread issues impacting communication, EFTPOS systems, public transport, and emergency calls.

Telstra executives confirmed that one of its three Network Time Protocol (NTP) servers designed to ensure its systems had the correct time reset to the year 2006 during routine maintenance. A technician had shut down and restarted the server in Melbourne to replace a faulty backup power feed in the server’s chassis. Due to an underlying software configuration issue, the server came back online believing it was 2006.

From there, the fault spread quietly. Over the next few hours, the incorrect date rippled across the network, causing authentication certificates on other servers to become invalid. Once those certificates were treated as expired or not yet valid, connected equipment began denying connections outright, blocking voice calls and data access for customers nationwide.

An avoidable failure

This is a striking example of how a single wrong date can cascade into a national telecommunications failure. The slow accumulation of small mistakes – a missed software update, an undocumented change, a maintenance procedure that didn’t reflect the equipment’s true state – was enough to bring down a national network.

This serves to highlight how incredibly important time and precision are in all aspects of our lives, especially in the digital world.

Why time has to be exact

Maintaining the correct time is intrinsic to a huge variety of systems and operations that keep our society functioning. This is why the National Measurement Institute (NMI) exists: to measure time in Australia and ensure it is extremely accurate.

The video below from NATA, the National Association of Testing Authorities, explains how time needs to be accurate to a millionth of a second just to let something as ordinary as a phone call in your car happen seamlessly.

IMG 20190408 092913 utd

What this means for testing & calibration

As we have seen with the Telstra outage, incorrect time measurements can cause cascading failures across systems that millions of people rely on every day. Ensuring the correct time – and being able to prove it – is something we take very seriously at SuperTest.

Our NATA-accredited facility conducts a wide range of tests in controlled environments, and as an externally accredited lab, we need to maintain and prove the validity of our data at every stage. That means our equipment, our processes, and our documentation all have to withstand external scrutiny.

Synchronising phones, computers, dataloggers and stopwatches helps us maintain accuracy with all test schedules carried out at SuperTest, both for our NATA-accredited tests and our bespoke programs. Just as importantly, we document changes to our equipment and procedures thoroughly, so that accuracy doesn’t depend on any one person’s memory of how a system is configured.

The Telstra outage is a timely reminder that ‘close enough’ isn’t good enough when it comes to time.

Whether it’s a millionth of a second or a properly documented maintenance record, the details that seem the most minor and insignificant are often the ones holding everything else together.

SuperTest: accuracy you can count on

Our NATA Accredited SuperTest facility ensures traceability, comparability, validity, impartiality, technical proficiency, and precision in our measurements – across every part of the process.

Search SuperCool

Find the best HVAC&R products for your mobile, industrial or commercial needs.