Microsoft builds Maia 300 for inference

Microsoft detailed Maia 300, its third-generation inference chip, and the numbers show where the gains come from. Each accelerator carries 288 GB of HBM4 at 18 TB/s, 2.5 times Maia 200, and 96 of them fill a 167 kW rack. Up to 128 racks link into a 12,288-accelerator superpod. Microsoft claims seven times Maia 200’s rack throughput and 40% more tokens per megawatt than GPU systems. We break down the design, test the claims, and list what to watch at Ignite.

Microsoft has built a chip for answering questions, and it plans to build many of them. Maia 300, which Microsoft announced on Monday, targets inference, the work an AI model performs each time it responds to a prompt. Production deployments begin in 2027. Microsoft pairs the launch with a multi-gigawatt rollout that it intends to make the primary inference system for its own products and services. A gigawatt equals roughly the output of a typical nuclear reactor, so the plan describes power capacity on the…