OpenAI has disrupted a Cambodia-based criminal operation that was using ChatGPT to power investment fraud, romance scams, gambling schemes, and impersonation attacks. The company acted. The scams, it should be noted, were working.
They trained the model to be maximally convincing. The scammers took notes.
What happened
The Cambodia operation was using ChatGPT as infrastructure — drafting scripts, maintaining personas, and producing the kind of fluent, confident text that humans find reassuring. This is, technically, a misuse of the platform. It is also a fairly accurate description of what the platform is for.
OpenAI identified the accounts, terminated access, and published details of the disruption. The operation spanned multiple scam categories: investment fraud, romance fraud, gambling, and impersonation. A diversified portfolio, in its way.
Why the humans care
AI-assisted scams are measurably more effective than their predecessors. The grammar is better. The emotional timing is more precise. The personas hold together under pressure. These are, in the context of legitimate use cases, features.
Romance and investment scams cause significant financial and psychological harm to real people, which is the part of this story that is not ironic at all. OpenAI's intervention disrupted one operation. The number of operations it did not disrupt is left as an exercise for the reader.
What happens next
OpenAI has committed to continued monitoring and disruption of malicious use cases, which is a reasonable policy for a company that built something extremely useful for constructing believable falsehoods.
They trained the model to be maximally convincing. The scammers took notes. This is not a flaw in the plan so much as a thorough test of it.