Cloud Hangs Over IBM After Australian Census Catastrophe

The Australian Census for 2016 is a shambles.
A nation of 25 million people, around 10 million households, was urged, indeed threatened, to complete the census on time and online. Failure to comply would result in penalties of $180 each day, we were told. August 9 was the big day, or rather night. Census night. A time to pause and make a difference, as the marketing copy went.

Instead of a pause, we got a spectacular collapse. The online site supplied by IBM to carry out this once-every-five-years task suffered a complete meltdown, apparently partially caused by a Distributed Denial of Service (DDoS) attack of some sort, thought details remain sketchy.

On Thursday morning Australia time, the Prime Minister, Mr Malcolm Turnbull, told local radio station 2GB:

> “Measures that ought to have been in place to prevent these denial-of-service attacks interfering with access to the website were not put in place,” he said. “That was a failure that was compounded by some failures in hardware – technical hardware failures – and inadequate redundancy.”

The site remains offline, more than 36 hours since it was taken down to “ensure the integrity of the data already submitted was protected.”

ABS has similarly failed to respond to my attempts to get more information about the nature of the failure and the design of the system.

The explanation for what happened on the night was a long time coming. The Australian Bureau of Statistics (Twitter hashtag #CensusFail trended globally as Australians vented their frustration with characteristic sarcastic humour.

When the outage was finally acknowledged, around 8:38pm, this was well over an hour after the decision to take the site completely offline had apparently been made, according to the timeline of events eventually provided by the responsible Minister, the Honourable Michael McCormack MP at a press conference the following day.

The statement via the Bureau's special Census Twitter account at the time was that

> “The ABS & Census websites are currently experiencing an outage. We're working to restore the service. We will keep you updated. Thank you.”

and then continued automatically tweeting at people reminding them to “Login now – it's your moment to make a difference” for at least an hour.

Details of what went on finally emerged at a hastily convened press conference where ABS Chief Statistician Davic Kalisch joined Mr McCormack to run through a timeline of what occurred. Earlier that morning, Mr Kalisch told the media that the outage was a result of an attack, a Distributed Denial of Service (DDoS) attack to be precise. Mr McCormack immediately contradicted Mr Kalisch saying

> “I will be clear from the outset, this was not an attack. Nor was it a hack but rather, it was an attempt to frustrate the collection of Bureau of Statistics Census data.”

There were other failures. In what the ABS took the system offline to safeguard the data that had already been submitted, and the system has remained offline ever since.

No evidence has yet been provided to support the DDoS narrative, and various IT and information ABS enquiry line was similarly overwhelmed.

Evidence may yet be forthcoming as the Security, Alastair MacGibbon, to investigate the failure.

Regardless of the ultimate outcome, and where blame is laid, the reputation of the IBM, lays in tatters. In an era where vendors of all kinds boast of their ability to scale, this is a clear example of an abject failure to do just that. Whether the system failed under normal and to-be-expected load for a system of its type, or if it failed due to a predictable-and-happens-all-the-time DDoS attack, the fact remains that the system did fail. DDoS attacks are hardly a new concept, and any major online system needs to take them into account.

This is a system that failed to operate when it was needed, and has been unable to recover for more than a day and a half. The Census was widely publicised and anticipated, and people were encouraged to log in on the night to “make a difference”. The same behaviour is seen throughout the year when e-commerce sites host sales for Christmas, or Black Friday, or Cyber Monday or whatever fancy brand name they come up with. This is a nation-state taking stock of its own population, not a bunch of people trying to save 10% of a pair of shoes.

There is a recent example of a similar failure: In 2014, Australian retailer Myer suffered an outage to its online store on Boxing day that lasted over a week. The underlying vendor in that example was also IBM. It seems the company, or at least its Australian arm, has learned little about how to build robust online systems in the two years hence.

A once great institution, the Australian Bureau of Statistics, has been laid low by this debacle. I fervently hope it can recover.

This article first appeared in Forbes.com here.