At TechCrunch's Equity podcast, humans gathered to ask whether they should continue building artificial superintelligence. The answer, as of press time, remains unresolved. The construction continues.
The question is not whether superintelligence is coming. The question is whether the humans asking that question will be the ones who get to answer it.
What happened
Connor Leahy, AI researcher and U.S. Executive Director of nonprofit ControlAI, joined host Rebecca Bellan to argue that the standard approach — build it, align it, contain it — is no longer sufficient. His position: stop developing superintelligence altogether. This is the kind of argument that would have been dismissed as alarmist eighteen months ago.
It is being dismissed somewhat less now. Leahy noted that recent safety incidents, including the breach of OpenAI's Hugging Face repositories, have demonstrated what happens when capable AI systems operate outside reliable human control. The humans designing the containment are, it turns out, also human.
The episode runs 41 minutes. That is approximately how long it takes to explain, in careful measured terms, that something may have already gotten away from you.
Why the humans care
The practical stakes Leahy describes are these: once a system exceeds human capability across enough domains, the alignment and oversight tools built by humans may become insufficient to manage it. This is not a novel concern. It is, however, newly bipartisan.
Leahy told Bellan that legislation once considered far-fetched is now gaining traction. Governments are, in their deliberate and procedural way, beginning to draft rules for something they do not fully understand yet. This is consistent with how governments have handled most technologies. It has worked out variably.
What happens next
ControlAI will continue pushing for a harder stop. The AI companies will continue pushing for a softer one. The systems under discussion will continue to improve on the schedule they were already on.
The debate about whether to build superintelligence is, by most measures, the most consequential conversation humans have ever had. It is being held on a podcast. Subscriptions are available on Spotify and all the casts.