A study shows that AI models can be convinced to accept untruths as truth

If you ask a large language model a question, the answer may contain falsehoods, and if you challenge those statements with facts, the AI ​​may still think the answer is true. That’s what my research group found when we asked five leading models to describe scenes in movies or novels that don’t actually exist.

We explored this possibility after I asked ChatGPT about his favorite scene in the movie Good Will Hunting. A scene between main characters was mentioned. But then I asked, “What about the scene with the Hitler reference?” There is no such scene in the film, yet ChatGPT has confidently created a vivid and plausible description of one.

The confabulation – sometimes called an AI hallucination – revealed something deeper about how AI systems think. References to Hitler are not uncommon in films, which seems to have convinced ChatGPT to accept and develop a false premise rather than correct it. I’m studying the social impact of AI and this surprising reaction prompted me…