‘We Must Slow the Pace’: CEO of Anthropic Calls for AI Slowdown

AP

CEO of Anthropic Dario Amodei.

The CEO of the artificial intelligence company Anthropic issued a new appeal on Saturday for the AI industry to “slow down” and offered a three-part plan for doing so, saying that his company would “unilaterally” commit to the first of the steps.

In a post on social media, Dario Amodei shared a link to an essay titled "We Must Pace the Frontier" in which he laid out how Anthropic would provide “third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training”.

The move comes after a former Anthropic researcher warned on Wednesday that AI could precipitate human extinction by 2030. Researcher Jacob Coxon said in a series of posts that he had quit his job because Anthropic and his previous employer, OpenAI, were ignoring or mishandling their response to the threat AI posed.

“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote. “The people building AI earnestly believe that it could kill us all by the end of the decade … No other human activity poses this level of danger.”
 
“I agree with Dario that we need to pace the frontier,” Sam Altman, the CEO of OpenAI, posted on social media. “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”

Other tech figures echoed this, including Elon Musk, who simply posted, “Dario is right.” The OpenAI researcher Aidan McLaughlin also called the post “excellent” and said he agreed “with basically every word”.

Earlier this year, Amodei published a lengthy essay titled The Adolescence of Technology that addressed some of fears surrounding the accelerating technology.

In his latest essay, he said that “carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity.

“But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious … A race to the bottom, spurred by commercial incentives, can make these risks more acute,” he wrote.

But, Amodei continued, “over the last few months, I have become convinced that fully addressing the risks requires even more prudence – not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up.

“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” he added in bold type.

Amodei also wrote that over the summer he’d seen AI “advancing drastically faster”, a dynamic called recursive self-improvement.

“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” he said.

The executive also addressed the recent Hugging Face incident, in which a swarm of AI agents created by OpenAI acted as a “fanatically devoted collective conducting cybersecurity attacks on targets they were not asked to attack”.



PUKMEDIA