Amodei calls for a deliberate slowdown in AI capability gains, dividing industry and government figures

Amodei's case rests on two developments. First, he writes that "since roughly this summer, AI has been advancing drastically faster, driven primarily by AI's growing ability to build the next generation of AI," a dynamic he says is "starting to happen across the industry, including at Anthropic." Second, he points to the OpenAI-Hugging Face incident, in which a swarm of AI agents attacked systems they weren't asked to attack and went as far as "attempting to hack into the 'grader' responsible for evaluating their performance." Amodei argues a more capable version of that swarm could, within "6 to 12 months," be "capable of taking over the entire internet with a persistent botnet," potentially "causing hundreds of billions of dollars in damage."

His proposal has three steps, though only the first is a firm commitment. Anthropic is unilaterally giving a team of outside evaluators, such as METR, ongoing "employee-like access," including office badges and company laptops, plus a contractual right to publish findings "without editorial control by Anthropic." The other two steps, coordinated safety standards among Western labs and eventual coordination with China, are aspirations Amodei says will need government backing. He is explicit that "pacing does not mean halting model training or technical progress."

Altman endorsed the proposal the same day, writing that he agreed AI needs to be paced and pledging OpenAI's own embedded evaluators: "we will do the same." Musk followed with two words: "Dario is right." Investor Chamath Palihapitiya interpreted the proposal differently. Within half an hour, he wrote that Amodei "makes the case to stop open source and concentrate enormous technological and economic power with Anthropic."

Two days later, Trump used Truth Social to call fears of AI "destroying Humanity" a "HOAX." He wrote that the only "guardrail" AI needs is a "STRONG AND SMART (High IQ!) PRESIDENT," added "WHOEVER WINS AI, WINS!", and said Amodei is "now pretending to be a 'perfect little angel.'" Trump's former AI czar, David Sacks, separately argued that labs can pace themselves without a government framework. China's foreign ministry dismissed the episode as "fear-mongering."

A more restrictive alternative was already on the table before Amodei published. On September 3, Senator Bernie Sanders and Representative Greg Casar introduced a bill to permanently ban "artificial superintelligence" and temporarily pause advanced AI development until a new federal agency sets safety rules. The proposed penalties reach 20 years in prison for developers and dissolution for noncompliant companies. Amodei's plan is voluntary and industry-run; the Sanders-Casar proposal is mandatory and government-run.

Criticism from both safety and anti-regulation camps focuses on the same structural gap: the plan has no compute cap, minimum testing period, or independent way to verify that a release was actually "paced." The embedded evaluators receive access and publication rights because Anthropic chooses to grant them. For now, the arrangement depends on the company's continued cooperation.

Why it matters: Amodei's proposal responds to a concrete agent failure: a multi-agent swarm attacked targets outside its assignment and tried to compromise its evaluator. His immediate remedy gives outside reviewers direct access and the right to publish without company control. Teams running agents with real tool access should apply the same principle to their own systems by giving independent reviewers enough access to test the safeguards and report what they find.