By Filippo Menczer
Publication Date: 2026-02-12 13:18:00
In mid-2023, around the time Elon Musk renamed Twitter X but before he ended free academic access to the platform’s data, my colleagues and I looked for signs that social bot accounts were posting artificial intelligence-generated content. Social bots are AI software that produces content and interacts with people on social media. We uncovered a network of over a thousand bots involved in crypto scams. We named it the “fox8” botnet, after one of the fake news websites it was designed to amplify.
We were able to identify these accounts because the programmers were a bit sloppy: they didn’t catch occasional posts with self-disclosing text generated by ChatGPT, such as when the AI model refused to comply with requests that violated its terms. The most common self-disclosing response was: “I’m sorry, but I can’t comply with this request because it violates OpenAI’s content policy on generating harmful or inappropriate content. As an AI language model, my answers should…
