AI starts improving itself faster than society can manage

• Australia is supporting a proposal led by AI company Anthropic for global “frontier” AI developers to agree on mechanisms to slow or pause development if systems begin improving themselves too rapidly.
• The plan does not call for an immediate halt but suggests safeguards if AI reaches a point where it can recursively improve beyond human control, which experts warn could pose significant risks.
• Officials emphasise that AI progress should not outpace safety and understanding, highlighting the need for coordinated international oversight as capabilities accelerate.

Australia is backing a call from leading AI company Anthropic to stop or slow down work on the technology if AI starts improving itself faster than society can manage.

Assistant Minister for Science, Technology and the Digital Economy Andrew Charlton said Australia welcomed efforts to make sure the world’s most powerful AI developers had agreed safety measures in place before the technology reached that point.

“The pace of progress must not outstrip understanding or undermine safety across society, and no company should develop AI that is unsafe,” he said.

The call from Anthropic, the company behind the Claude chatbot, stops short of demanding an immediate halt to AI development.

Instead, it says that the “frontier” AI labs — organisations like Anthropic, OpenAI and Google that are developing cutting-edge technology —should agree on a verifiable way to slow or temporarily pause work if advanced systems begin showing signs they can rapidly improve themselves beyond what humans can do.

In a blog post published this morning Australian time, Anthropic says self-recursive development is “not there yet” and the outcome is “not inevitable”, but argues it could happen sooner than governments and public institutions are ready for.

The company said more than 80 per cent of the code merged into Anthropic’s codebase was authored by Claude as of May 2026. It also said its engineers were shipping about eight times as much code per quarter as they did from 2021 to 2025.

Anthropic said that, while AI systems that could improve themselves could have great benefits, they also could “increase the risks of humans losing control over AI systems”. …

... read the full article here

Original article and photo by "ABC News"