Anthropic and OpenAI employees speak out about humanity’s extinction as debate reaches fever pitch

• An Anthropic researcher’s resignation prompted a fierce debate over the dangers of AI acceleration.
• Some AI company CEOs have been outspoken about the risk of an AI catastrophe.
• Here is what employees at Anthropic and OpenAI are saying about AI safety measures.

Jacob Coxon, former Anthropic AI researcher”If we build superintelligence without more robust alignment I expect we will permanently lose control of it. If that happens then most people could die,” Christiano said after OpenAI announced he was joining its board and Safety and Security Committee.

“I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term,” Christiano added. “I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.”

Chris Hayduk, life sciences researcher at OpenAI

Chris Hayduk, a life sciences researcher at OpenAI, reposted Coxon’s post but is looking at the brighter side.

“I work at OpenAI and personally think AI has been and will continue to be an extremely beneficial technology to humanity,” Hayduk wrote on X on Thursday.

“The conversation should be around how many billions of lives it will save,” Hayduk added.

Tomek Korbak, AI safety researcher at OpenAI

Tomek Korbak, an AI safety researcher at OpenAI who previously worked with Coxon, wrote in a post on X that Coxon remains a “very thoughtful researcher.”

“Neither anthropic nor openai are on track to solve alignment to a degree sufficient for shipping superintelligence and we need to slow down,” Korbak said on Thursday.

Jen Leike, member of technical staff at Anthropic

Jen Leike, an AI researcher at Anthropic, said in a post on X on Thursday that the most effective safety measures take time to implement, such as the “jailbreaking mitigations for Opus 4” that took over a year to develop.

“Now is a good time to build institutional mechanisms to pace the frontier of AI development,” Leike said.

“The industry is locked into an all-out scaling race to build superintelligence as quickly as possible, and we may need to give everyone more time for safety and alignment mitigations,” Leike added.

Julie Steele, member of technical staff on OpenAI’s preparedness team

Julie Steele, who works on OpenAI’s preparedness team, reposted Coxon’s post on X.

“I work at OpenAI,” said Steele on Wednesday on X. “In my personal capacity, I also think we need to slow down.”

Samuel Marks, technical staff at Anthropic

Samuel Marks, a technical staff member at Anthropic, wrote about Coxon’s warning in an X post on Wednesday. He said he was writing in a personal capacity, not on behalf of Anthropic.

He said that AI developers believe their technology could cause human extinction, and “the more senior the employee, the more concerned they are.”

AI models frequently “severely misbehave,” he said, and while developers have methods to “nudge” them toward better behavior, they cannot be robustly aligned.

“I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes,” Marks wrote.