By Luo Mai
Publication Date: 2025-11-20 14:27:00
It is becoming increasingly difficult to get today’s artificial intelligence (AI) systems running at the scale required for further advances. They require enormous amounts of memory to ensure that all of their processing chips can quickly share all the data they generate to function as a unit.
The chips that have driven the deep learning boom in the last decade are called graphics processing units (GPUs). They were originally designed for games, not AI models, where every step of their thought process must occur in significantly less than a millisecond.
Each chip contains only a modest amount of memory, so the large language models (LLMs) that underlie our AI systems must be split across many GPUs connected via high-speed networks. LLMs work by training an AI on massive amounts of text, and every part of it involves transferring data between chips – a process that is not only slow and energy intensive, but also requires more and more chips as models grow.
For…
