Teaching LLMs how to count

The transformer architecture underlying today’s large language models has challenged our assumptions of what AI can do. Code-writing, language translation, and synthesizing information from the far corners of the web — these are all tasks that LLMs today can seemingly handle with ease. In a few short years, it’s changed how we work.

But transformer-based models have inherent limitations that researchers are bumping up against as they try to extend their capabilities and lower their computational costs. One of the transformer’s core flaws, at least theoretically, lies in its limited capacity to model sequential data. This can make some kinds of counting problems — like tallying the number of “r”s in “strawberry” — surprisingly difficult.

By breaking the problem into steps through chain-of-thought (CoT) prompting or switching to a “universal” architecture with more sequential processing, a transformer-based model can hit on the right answer. But workarounds like…