Australian Robodebt scandal shows the risk of rule by algorithm


Tourists at Australia’s Parliament House in Canberra. Australia’s Robodebt scheme, whose algorithms wrongly calculated overpayments to 400,000 welfare recipients, is being investigated. — Reuters

MELBOURNE: When Australia’s welfare agency mailed Nathan Kearney to say he owed it thousands of dollars, the musician knew something was up. But try as he might, Kearney couldn’t reach anyone human in government to put right the horrible wrong.

That was back in 2016, the start of a five-year ordeal.

First – a slew of harassing phone calls from debt collectors. Then his tax rebate was withheld to pay arrears of about A$2,000 (RM5,933) that government said he owed.

A year after the first debt notice, he was mailed another, telling him he owed about A$4,000 (RM11,867).

“It was just really beyond my comprehension, how they could be asking me for that money," said Kearney, 33, who had to leave his shared house in Brisbane and move back to his parents’ home hundreds of miles away.

“It was the only way that I could imagine being able to cope financially,” said Kearney, whose debts spiralled even higher after he bought countless sessions of therapy to weather an ordeal that had left him “really, really depressed”.

“It took me five years to get back on my feet,” he told the Thomson Reuters Foundation.

Kearney is one of about 400,000 welfare recipients in Australia who were wrongly accused of misreporting their income to the welfare agency, and consequently slapped with fines.

The errors were the result of an automated debt recovery scheme, known as Robodebt, set up by the former conservative coalition government, which wrongly calculated that the welfare recipients owed money and so issued a ream of debt notices.

Running from July 2015 to November 2019, the scheme used algorithms to calculate the overpayments, raising more than A$1.7bil (RM5.04bil), which the government was forced to repay or wipe when a court ruled the scheme unlawful in 2019.

A Royal Commission, the most powerful type of government inquiry, is now investigating the scheme, with bureaucrats, ex-ministers and victims all due to give evidence.

When the Labor government announced the inquiry in August, Government Services Minister Bill Shorten called Robodebt a “shameful chapter in the history of public administration”, and a “massive failure of policy and law”.

Intervention can cut Robodebt errors

Governments around the world have turned to artificial intelligence (AI) to process bulging welfare claims more quickly and to reduce fraud. But the technology has been plagued by operational issues and charges of bias.

A United Nations expert warned in 2019 of the emergence of the “digital welfare state”, designed to slash welfare spending and embed government surveillance.

Some countries have pulled their schemes following errors.

In 2020, the Netherlands abandoned its automated system to detect welfare fraud after a court ruled it violated human rights. The campaign targeted low-income families, official reports show, disproportionately hitting ethnic minorities.

The British government has also come under scrutiny for automating its benefits system, which human rights groups say could push vulnerable claimants further into hunger and debt.

Such schemes are “inherently problematic”, and often fail because they lack all humanity, said Tapani Rinta-Kahila, a professor at the University of Queensland who studies the use of AI in the public sector.

“No matter how good an algorithm you have, no matter how sophisticated it is, it still cannot address fundamental human issues as sensitive as welfare,” he said.

“I’m not against using AI algorithms in making these services more efficient and work better, but there should be a high level of discretion on how to go about it – having human supervision, human intervention,” he added.

The Royal Commission found the government had received legal advice as early as March 2019 that the Robodebt scheme was probably unlawful, but didn’t stop it until November that year.

In 2020, then-Prime Minister Scott Morrison apologised in parliament for any “hurt or harm” caused, while defending the “difficult job” of recovering welfare debts.

But the scheme ran for too long and the government “kept on defending a system that was flawed”, said Rinta-Kahila.

Checks and balances vital

The Commission runs until March, its hearings livestreamed.

One victim, Melanie Klieve, said her experience with Robodebt made her feel like a “criminal”.

The spa supervisor received notice in 2016 that she owed about A$2,500 (RM7,416), and had her welfare payments reduced to pay off the apparent debt, putting her in financial distress.

“I rang them and said I cannot live on the amount that you’re giving me ... and they said there’s nothing we can do, it is what it is,” she told the Thomson Reuters Foundation.

So the 49-year-old sold her car, used food and petrol vouchers from the Salvation Army, and borrowed from her parents just to get by.

“It was pretty bleak – I stayed in my flat because I couldn’t go out, couldn’t afford food or anything.”

She said that “checks and balances” were vital to prevent a repeat Robodebt.

“I’d like to see that it never, ever happens to any vulnerable people,” she said.

Kearney, who has chosen not to appear at the Royal Commission, called the scheme “irrevocably evil” but doesn’t blame the technology.

“I don’t think it’s the computer system, that’s the worst part of it,” he said, lambasting government for issuing debt notices despite repeated warnings its system was broken.

“I think it’s the people who created the computer system and who allowed it to be used. It is the people.” – Thomson Reuters Foundation

Follow us on our official WhatsApp channel for breaking news alerts and key updates!
   

Next In Tech News

Disney nears tipping point as streaming profits start to offset cable decline
KKR raises tender offer price for Fuji Soft acquisition
Putin will look into YouTube disruptions, Kremlin says
Singapore clears South Korean Hanwha's buyout of Dyna-Mac
Samsung Electronics plans $7.2 billion buyback after share price plunges
Man who stole and laundered roughly US$1bil in bitcoin is sentenced to 5 years in prison
Stiff business: Berlin startup will freeze your corpse for monthly fee
White supremacist, Nazi content spread on Steam game service
After Trump’s victory, US election falsehoods shift left
Review fatigue? Consumers are less likely to share feedback after a purchase

Others Also Read