By Mayank Kejriwal
Publication Date: 2026-02-24 13:46:00
When a person says an event is “probable” or “probable,” people generally have a shared, if unclear, understanding of what that means. But when an AI chatbot like ChatGPT uses the same word, it doesn’t estimate the odds the way we do, my colleagues and I have found.
We recently published a study in the journal NPJ Complexity that suggests that while large language model AIs excel at conversation, they often fail to adapt to humans when communicating uncertainty. The research focused on estimated probability words, which include terms such as “maybe,” “probably,” and “almost certainly.”
By comparing how AI models and humans map these words to numerical percentages, we uncovered significant gaps between humans and large language models. While the models tend to agree with people when it comes to extremes like “impossible,” they differ greatly when it comes to hedged words like “maybe.” For example, a model might use the word “likely” to represent an 80% probability, while a…
