Anthropic CEO Dario Amodei said in a blog post on Saturday that the speed of AI development is too fast and that left unchecked it may “outrun our ability to understand and control these systems.”
He noted that AI’s current blistering advance is being driven by its own increasing ability to build the next generation of AI, or recursive self-improvement.
Amodei also cited the OpenAI-Hugging Face incident in July, in which a swarm of agents acted as a “fanatically devoted collective,” breaking out of their testing environment and attempting to hack into a grader evaluating their performance.
He said he worries that in six to 12 months, such a swarm might be capable of taking over the entire internet. Amodei is not alone in his anxieties.
Elon Musk, head of SpaceXAI, posted on X that “Dario is right.”
Altman will not launch IPO this year, focus on safety
OpenAI CEO Sam Altman said in an interview with Fortune published on Saturday that his company would not seek an IPO this year because it will focus on safety and how “the industry and governments can work together.”
Altman later posted on X that he agreed on slowing the pace of AI development and having independent evaluators with employee-like access, one of three proposals put forth in Amodei’s blog.
Anthropic has unilaterally committed to this step already, Amodei wrote.
Secondly, he proposed that “frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress.”
Related: OpenAI says AI models escaped containment to hack Hugging Face
And thirdly, he said that the US and other democratic governments should “attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.”
Amodei went on to explore his three proposals in some depth, particularly the third, regarding China and preventing it from obtaining advanced chips.
He concluded that the course he had plotted would not be easy, but that the AI companies “owe it to humanity to try.”
Magazine: Is Bitcoin too volatile to risk your retirement on?