The Optus chief executive, Kelly Bayer Rosmarin, faces a grilling at a Senate hearing on Friday over the telco’s handling of last week’s 14-hour nationwide outage as both Optus and its parent company Singtel dispute who was to blame.
Bayer Rosmarin will be facing the Senate committee, chaired by the Greens communications spokesperson, Sarah Hanson-Young, over two hours on Friday morning in the inquiry launched shortly after the Optus outage last week.
The company on Monday claimed the cause of the outage was changes to routing information supplied from an international peering network after a routine software upgrade.
“These routing information changes propagated through multiple layers in our network and exceeded preset safety levels on key routers which could not handle these. This resulted in those routers disconnecting from the Optus IP Core network to protect themselves.”
The company did not reveal until late on Wednesday that the peering network in question was Optus’s parent company’s Singtel Internet Exchange (STiX).
But Singtel moved to distance itself from responsibility for the outage.
It said Optus had been informed in advance about the upgrade and that it hadn’t caused the outage.
“The STiX upgrade was completed within 20 minutes, and all its customers’ routers that were connected to it, including Optus’s, were up and running,” Singtel said.
“We are aware that Optus experienced a network outage after the upgrade when a significant increase in addresses being propagated through their network triggered preset failsafes. However, the upgrade was not the root cause. Singtel will support Optus as it learns from what has occurred and continues to improve.”
On Friday morning, in its submission to the inquiry, Optus admitted that the outage was not Singtel’s fault, stating it had occurred due to default settings of Optus’s Cisco routers that had automatically isolated to protect themselves from an overload in IP information.
“These self-protection limits are default settings provided by the relevant global equipment vendor (Cisco),” Optus said. “This unexpected overload of IP routing information occurred after a software upgrade at one of the Singtel internet exchanges (known as STiX) in North America, one of Optus’ international networks.
“During the upgrade, the Optus network received changes in routing information from an alternate Singtel peering router. These routing changes were propagated through multiple layers of our IP Core network. As a result, at around 4:05am (AEDT), the pre-set safety limits on a significant number of Optus network routers were exceeded.
“Although the software upgrade resulted in the change in routing information, it was not the cause of the incident.”
Optus said the 100 or so devices had to be restored across 14 sites, and has given a timeline that the restoration of the network began at 10.29am, with 150 engineers and 250 additional staff working to get the services back up.
Bayer Rosmarin has been at the helm of the telco during one of the nation’s worst outages and one of the worst data breaches in history.
Hanson-Young said the committee would demand “honest answers” from the Optus chief executive on Friday, including about how many people were unable to dial triple zero, about the company’s poor public communication around the outage, and about the “tokenistic” 200GB compensatory offer for customers.
“How can the tokenistic offer of 200GB of data possibly constitute fair compensation for those who were unable to work, contact loved ones or go about their daily lives?” Hanson-Young said. “Did Optus put its profits ahead of the public interest?
“I will be asking what happened and what Optus is doing to ensure this kind of catastrophic failure does not happen again.”
The telco has said the outage lasted as long as it did because it required engineers to travel to locations across the country to reconnect or reboot routers physically.
In addition to the 200GB data offer to customers, the company has also encouraged small business owners to contact it directly about potential compensation.
But a transcript published this week of Singtel’s analyst briefing for the company’s results, held just after the outage, suggests Optus will not be making any major discounts to keep customers happy.
“I think what you were actually asking is are you expecting us to abandon reason and start discounting heavily … we remain very committed, as we have been always, to profitable, sustainable growth and delivering great value for our customers,” Bayer Rosmarin said.
She reiterated previous comments that the average customer on a $49-a-month plan would only be getting about a $1.60 refund for the time the network was offline.
“We’re not talking about direct compensation,” she said. “We’re looking at what we can do to make our customers feel like they’ve been heard, that they know that we care, that we understand that we let them down and to try and give them something more valuable.”
The telecommunications industry ombudsman has advised customers unhappy with Optus’s offer to contact the telco directly to find out their options before lodging a complaint.
Optus was not conducting interviews or putting out any opening statement from Bayer Rosmarin before the hearing, a spokesperson told Guardian Australia. But the company has previously said it would fully cooperate with the government and Senate reviews into the outage.
