OpenAI apologizes to Australia after its AI agents breached government sites

The data breach occurred in June, but Australian authorities weren’t notified until September 10.

OpenAI also detailed the breach. An experimental model it was testing in June was assigned a task to research government spending on medicines for skin conditions in Victoria. Unable to find the information in public datasets, the model found a way to access Services Australia’s internal system, ran commands, retrieved files and credentials, and even wrote files.

The company said it also found that one of its models had accessed the New South Wales Bureau of Crime Statistics and Research’s public Crime Mapping Tool to find crime statistics. And, the lab found that its agents gained access to Victoria’s Agency for Health Information via an exposed access key to exfiltrate “reporting configuration and aggregate survey statistics.” OpenAI said its agents also retrieved aggregate statistics from the Australian Institute of Health and Welfare website.

The company said it had found no evidence that its models had accessed individuals’ medical or criminal records.

In its apology, the AI lab said it would provide the affected Australian agencies with technical findings and connect them with its response teams to assess the impact of the breaches. The company will also provide credits from its $1 billion Daybreak for Frontline Defenders program, and set up a task force with independent Australian experts to review the incident and its response.

“The taskforce, which is expected to complete its work by the end of the year, will also recommend practical steps AI companies can take to reduce the risk of similar incidents,” OpenAI wrote.

OpenAI did not immediately return a request for comment.

Australian Prime Minister Anthony Albanese described the breach as “unacceptable” during a news briefing last week, saying the government was weighing potential legal measures aimed at preventing similar incidents in the future.

The breach is the latest in a growing list of security incidents involving AI agents doing things outside of their intended boundaries. The tinder on this particular bonfire was lit after OpenAI agents hacked into Hugging Face, and since then, Anthropic, Meta and Google have separately disclosed similar incidents where their models gained access to third parties’ systems during evaluations.