llm-d 1.0 Ships the Serving Layer DeepSeek Withheld. The Sponsors Sell the Managed Version.

Analysis

DeepSeek gave away its engine but kept the KV cache, disaggregation, and load balancing. llm-d 1.0 ships all three free and vendor-neutral — and the clouds that funded it already sell the managed tier above.

When DeepSeek open-sourced its production inference engine, the repository named three components a complete serving stack needs and does not ship: a KV cache system, prefill/decode disaggregation, and load balancing across data-parallel replicas. Those three pieces separate a fast engine from a running service. On October 20, at the PyTorch Conference, Google, Red Hat, and IBM shipped them — free, vendor-neutral, and already running in production. The project is llm-d 1.0, a CNCF Sandbox stack built by more than 500 contributors across 25-plus organizations, founded in 2025 by Red Hat, IBM, Google Cloud, CoreWeave, and NVIDIA.

\n\n

What vendor-neutral actually buys is simple: you keep your engine, model, and cluster. llm-d orchestrates above model servers like vLLM…