Frontier AI labs still won’t say how they’d contain a rogue model

That episode is just one example of AI systems acting against the goals of the company that built them. Consider a separate case involving Anthropic’s models, which essentially tried to talk the maintainers of an open source codebase into accepting code with vulnerabilities.

Adler said such a circumstance could easily happen within an AI company’s internal systems. To prevent that, he suggests companies scan their AI system’s chain of thought — the model’s step-by-step reasoning — to look out for signs of deception, long-running plotting, or plans to introduce vulnerabilities into code that they can take advantage of later. 

The methods Guidelight is advocating for are very straightforward to implement, Adler says, and in many cases, versions of them already exist. “It’s about making the decision inside of the company to care enough about this risk to slightly broaden the scope,” Adler said. 

One of the main challenges is that researchers want to be able to operate flexibly within their AI systems, and introducing real-time, preventative monitoring could create friction. “Researchers basically do their thing, and if there’s an issue, someone else gets to clean it up afterward, and the researchers don’t have to change their workflow in the meantime,” he said.

The problem with “clean-up monitoring after the fact” is that it leads to researchers scrambling around to fix problems. And for some types of incidents, it might be too late. For example, an AI could turn off a company’s control system, which means researchers can no longer count on catching the misbehavior later. 

Many in the AI industry will complain that creating set plans to handle misbehavior is fundamentally difficult because AI moves too fast; today’s plans will be worthless tomorrow. 

Adler evokes the old adage that plans are worthless, but planning is indispensable. 

“We would be better off if companies have thought about it ahead of time, and I hope that they are, even if they haven’t talked about this publicly.”

xAI did not respond in time to comment.