Perplexity builds custom serving infrastructure to make AI search faster and cheaper

Running an AI search engine that fields roughly 400 million queries a month is expensive. Perplexity AI has been quietly building the infrastructure to make it less so, publishing research on custom serving technology that spans everything from compact embedding models to a proprietary engine capable of handling trillion-parameter models.

The embedding play

At the core of Perplexity’s retrieval improvements is the pplx-embed family of models, which includes pplx-embed-v1 and pplx-embed-context-v1. These come in two sizes: a 0.6 billion parameter variant and a larger 4 billion parameter version. Both were released in February 2026.

Through quantization techniques, which essentially reduce the numerical precision of model weights without destroying accuracy, Perplexity claims up to 32x reductions in storage requirements. Both models were optimized specifically for low-storage deployment, a design choice that reflects the practical reality of web-scale retrieval.

ROSE and custom networking

The more ambitious piece of infrastructure is what Perplexity calls ROSE, short for Runtime-Optimized Serving Engine. Introduced around February 2025, ROSE is designed to serve a wide range of model architectures, including the increasingly popular Mixture-of-Experts (MoE) format that powers many of today’s largest language…