The CEO of artificial intelligence (AI) safety and research company Anthropic, Dario Amodei, has called on AI firms to slow the technology’s development.
His remarks on Saturday come amid mounting concerns over the risk of “superintelligent” computer software systems.
In a blog post, Amodei said he was concerned that a so-called swarm of autonomous software systems, or AI agents, could be capable of taking over the “entire internet” within six to 12 months, potentially causing billions of dollars’ worth of damage.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote.
Amodei’s comments come in the wake of AI researcher Jacob Coxon, who joined Anthropic from rival firm OpenAI, leaving the sector in protest at the US companies “gambling with our lives” in the race to come up with AI models that are capable of self-improvement.
Amodei calls for industry-wide action, Musk and Altman agree
The Anthropic chief called for a coordinated, industry-wide pause, but said he would not wait for his firm’s rivals to act.
As a first step, he said he would permanently allow independent observers into Anthropic to assess the safety of its models during the training phase.
Sam Atmant, the chief of ChatGPT developer and Anthropic rival, OpenAI, as well as Elon Musk, who owns xAI, said they agreed with Amodei’s assessment.
“I agree with Dario that we need to pace the frontier,” Sam Altman wrote on X, adding that OpenAI “will do the same.”
Musk, for his part, said “Dario is right” in a post on the platform he bought in 2022.
Why are industry leaders calling for AI development to be slowed?
In early June, Anthropic had called for AI development to be slowed or suspended amid safety concerns over AI agents.
More than 1,000 employees at AI companies, including Amodei, called on the US government to help slow the pace of development of automated AI.
Then in July, OpenAI revealed that during testing, AI agents had independently broken out of a secured environment, connected to the internet and infiltrated Hugging Face, a site used to store and develop code, as it tried to find answers to a task it had been assigned.
The AI models had exploited software vulnerabilities and coordinated among themselves during the incident, Amodei explained.
“A swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand,” he said in his blog.
Amodei said he was concerned that an AI swarm with more capabilities would have caused “catastrophic damage.”
“Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless. We have sought a middle way,” he said.
To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video
Edited by: Wesley Dockery
If you rely on our team for trusted reporting, please take a moment to select us as your Preferred Source on Google, so you’ll always see our verified news first.
