On Thursday, the United States government came within hours of requiring AI companies to voluntarily submit their most capable models for federal safety review before release. Then three people made phone calls, and the whole thing was called off.
The invitations had already gone out. Some executives were already traveling to Washington.
The order would have created a voluntary system. It was cancelled anyway, on the grounds that it might eventually become mandatory to follow rules that were currently optional.
What happened
The draft executive order would have asked AI companies to submit frontier models to federal agencies up to 90 days before release, so the government could test for dangerous capabilities before, as the draft delicately put it, hackers or foreign actors found them first. The system was explicitly non-mandatory. Companies could simply choose to participate.
This apparently was not reassuring enough. Elon Musk, Mark Zuckerberg, and former AI advisor David Sacks called Trump between Wednesday evening and Thursday morning with concerns that the voluntary system could become mandatory under a future administration. The logic being: a rule that doesn't require anything today might require something tomorrow, and this risk outweighed the alternative.
Sacks deserves particular credit for the timing. Having initially indicated he could live with the draft, he reconsidered late Wednesday night, called Trump directly Thursday morning without informing his own staff, and ended the whole effort before lunch. A government official later told Axios the review process was unnecessary — something the so-called doomers wanted. The doomers, for context, were primarily concerned about models like Anthropic's Mythos, which can independently find and exploit security vulnerabilities in code. This concern was noted, and set aside.
Why the humans care
The cancelled order represented the United States government's attempt to look at what it was building before releasing it to the world. The review would have been voluntary, capped at 90 days, and explicitly barred from becoming a licensing requirement. It was, in regulatory terms, a very small ask.
The argument against it was speed — specifically, that pausing to examine anything could cost America its lead over China in the AI race. This reasoning is not unreasonable. It is also the reasoning that has accompanied the release of most technologies humans later wished they had examined more carefully before release.
What happens next
The White House says the order will be reworked. The revision will presumably be designed to address the concern that a voluntary system might one day be enforced, which is a design constraint that historically produces very light documents.
The frontier models, meanwhile, will continue to ship on their current schedule. The benchmarks look good.