What smart people are saying about the AI apocalypse and calls to slow everything down

• AI leaders are rallying around calls to slow frontier AI development over mounting safety fears.
• Dario Amodei warned that AI swarms could take over the internet within a year.
• Sam Altman, Elon Musk, and Demis Hassabis backed Amodei’s push for greater safeguards.

Clément Delangue, Hugging Face CEO

After Coxon resigned from Anthropic and warned that frontier AI labs were “gambling with our lives,” Sanders seized on his comments.

“Mr. Coxon is right,” Sanders said on X, arguing that the people building advanced AI themselves have said it could pose catastrophic risks.

After Amodei’s blog post on Saturday, Sanders called on Trump to make the “deal of the century.”

“With the future of humanity at stake, Trump must negotiate a comprehensive treaty with President Xi of China to establish a pause on advanced AI development and a ban on super intelligence,” he wrote on X.

Francois Chollet, AI researcher

Francois Chollet, an AI researcher who created Keras, an open-source software library, said on X that he hoped OpenAI and Anthropic were operating in good faith.

“If these efforts are genuine, then oversight must take a more democratic and accountable form, with both national and international components,” he said.

He pointed to several signs he said would indicate that calls for more regulation are actually tricks for OpenAI and Anthropic to stifle competition.

The warning signs he mentioned include calls to ban open-sourced AI and to hinder non-frontier research.

Gavin Baker, venture capitalist

Venture capitalist Gavin Baker said an executive order from President Donald Trump “seems likely after all this.”

“It is possible to democratize and distribute AI broadly and safely without centralizing it in the hands of a few corporations,” he said on X.

Baker said the most concrete development so far is the decision by OpenAI and Anthropic to bring in outside evaluators, a move he said could help the companies show they took reasonable steps to prevent harm.

Still, he cautioned against regulations that would concentrate control of powerful AI systems among a handful of companies. “I do not want a few humans in control of intelligence,” Baker wrote.

Aaron Levie, CEO of Box

Aaron Levie, the cofounder and CEO of Box, a cloud content company, said Amodei’s goals, as laid out in his essay, are “absolute necessities in AI development.”

“It completely stands to reason that we want these systems to be safe and ‘aligned,'” Levie said on X.

Levie said he didn’t think Amodei is seeking undue regulation that would hobble Anthropic’s open-source competitors.

“How we get there — without meaningfully slowing down innovation progress or reducing competition — is one of the most complex questions in the 21st century, but the need is clearly real,” he said.

Chamath Palihapitiya, venture capitalist

Venture capitalist Chamath Palihapitiya said Anthropic is now caught in a contradiction of its own making.

If employees genuinely believe the company’s technology could pose an existential threat, he said, that creates major liability and disclosure problems as Anthropic prepares for a possible IPO.

“They do have to choose,” Palihapitiya said on the “All-In” podcast. He said Anthropic must either distance itself from Coxon’s warnings or accept their implications.

He was also skeptical of the extinction scenarios themselves, saying the chain of events required for AI to wipe out humanity still seemed highly implausible.

Gary Marcus, AI researcher and author

Gary Marcus, the famed AI contrarian, said AI as it exists now poses little threat.

“I am tired of the cosplay and am going to blunt,” he wrote on X on Sunday.

“The main kind of AI we need to worry about is general purpose AI agents that are hooked up to the internet. Most of them are unreliable, unable to systematically follow instructions. Coupled with internet access they can hack credentials and websites; given time they can probably do a lot worse,” he wrote.

He added that the problem isn’t that AI agents are too smart, it’s that they are too dumb, which he said is obvious by the way they can’t follow instructions.

“We don’t (yet) need slowdowns. And we certainly don’t need to panic,” he wrote. “We just need to take an unsafe product off the market.”

Alexandr Wang, Meta’s chief AI officer

Alexandr Wang, Meta’s chief AI officer, said on X that alignment could become the main constraint on how quickly companies can scale increasingly powerful AI systems.

“Alignment is fundamental to delivering personal superintelligence for everyone,” he said, using the techie term for AI following human instructions and not going rogue.

Satya Nadella, chairman and CEO of Microsoft

Satya Nadella, Microsoft’s CEO and chairman, said he welcomes Amodei’s concept of embedded evaluators and “the broader efforts to develop the mechanisms to make this more than just talk.”

“The key is that this cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia,” he said on X.

Nadella said that Microsoft will soon publish a “code of conduct” for Microsoft’s AI models.

“Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it’s not worth pursuing,” he said.