The debate around AI governance heated up this past week as AI experts warned that the growing number of rogue AI agents and instances of misaligned behavior are merely a preface to worse threats in the future.
On Sept. 12, Anthropic CEO Dario Amodei warned in a widely read column that AI could soon slip the harness of human control if development is left unchecked. Within a few days, Microsoft delivered a Humanist AI Code of Conduct (read: manifesto) pledging human-centered development, and Google DeepMind's CEO agreed in principle in a post on X to the calls for more controls. Although Anthropic, OpenAI, and x.AI have all also suggested tighter government regulations for the technology, President Donald Trump dismissed the idea, suggesting that winning the AI competition with China is more important, reportedly saying, "Whoever wins AI, wins."
In his column, Anthropic's Amodei argued that model makers and AI development labs have a narrow window to refocus on safety and security.
"The current models are an almost endless goldmine of insight into both how to build AI well, and what can sometimes go wrong with it if it isn't built well," he wrote. "I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong."
In the meantime, companies are left to their own devices when it comes to implementing safety protocols for their agentic and autonomous AI programs. The incidents and the calls for regulations highlight the gaps between quickly advancing capabilities and the industry's ability to prevent harm, Ramy Rahman, senior principal solutions engineer at ArmorCode, tells Dark Reading.
"'Move fast and ask forgiveness later' is not an acceptable approach when organizations bear the consequences," he says. "Those developing and deploying AI should be accountable for the risks they introduce."