What early signs suggest may have gone wrong at Optus

Spread the love


That’s one of the tasks performed by BGP. It’s how networks share the addresses that are on their network, with other networks. Network providers like Optus run a mesh of BGP routers, that keep track of what addresses need to be forwarded to what overseas and local networks (remember, the internet isn’t one network, but a network of networks), and share their own addresses with other BGP routers around the world.

Orange isn’t a colour you want to see on your internet company’s dashboard. John Davidson

BGP is at the heart of the internet, but it’s notoriously tricky to get right.

In March last year, Twitter (now X) suffered an 45-minute outage when the Russian internet service provider (ISP) RTComm misconfigured its BGP routers so they “announced” to the world that they were now the official destination for some of Twitter’s European internet addresses.

Traffic that was meant to go to Twitter’s servers ended up in Russia, where it hit a dead end and gave Twitter users an error until RTComm announced a correction, and that correction propagated to BGP routers around the world.

At the time, Cisco System’s network intelligence company ThousandEyes said it was hard to know whether RTComm had deliberately “hijacked” Twitter’s traffic, or if it was yet another case of an accidental BGP misconfiguration.

“It’s important to understand that accidental misconfiguration of BGP is not uncommon, and given the ISP’s withdrawal of the route, it’s likely that RTComm did not intend to cause a globally impacting disruption to Twitter’s service,” ThousandEyes wrote in a blog.

“That said, localised manipulation of BGP has been used by ISPs in certain regions to block traffic based on local access policies,” the company wrote.

One way that companies such as Optus guard against the deliberate or even accidental hijacking of their internet addresses is to use cryptographic signatures on their routing announcements, so that when another network provider receives an announcement that traffic with a certain address is meant to be routed to a certain network, they can check that the announcement has been properly signed by the authorised owner of that address.

That signature system, known as Resource Public Key Infrastructure (RPKI), was what was glowing orange in dashboards around the world Wednesday, whenever anyone called up Optus’s network status.

The RPKI status for one part of Optus’s network showed dozens of misconfigured routes, as pictured above.

At this stage, whether it was Optus that accidentally misconfigured its own BGP routers, whether someone else did it accidentally, or whether someone else did it deliberately, remains to be seen.

It’s also not yet clear whether Optus’s BGP problems are the cause of its outage, or a symptom of its outage.

Stay tuned.



Source link