By Roger Montti
Publication Date: 2026-08-17 11:11:00
Google published a new research paper that found that frontier LLMs encode 95–98% of the tested facts but are unable to directly recall 26–34% in answers to queries. Part of the problem is that recall becomes more difficult when questions reverse the subject/object entity order in which a fact was encountered in training.
Parametric Information
Parametric information is, essentially, the information that LLMs have encoded during training. That information comes from the web pages, song lyrics, books, instructions, code, and everything else that the LLM was trained on.
The question the researchers were seeking to answer was: Why do LLMs fail to recall some of the information they were trained on? It was previously thought that maybe LLMs weren’t trained on enough information, but the researchers found that isn’t always the case for frontier LLMs.
The researchers explain that encoding is saturated, meaning that the information needed to answer questions is generally already…
