IBM-backed llm-d framework adds controls for running AI models in-house – SiliconANGLE

IBM Corp. and several other contributors to the open-source llm-d project are releasing version 1.0 of their distributed artificial intelligence inference framework, adding operational controls intended to help enterprises run large models reliably on their own infrastructure.

The release, announced ahead of the PyTorch Conference North America in San Jose today and tomorrow, focuses on managing competing workloads, maintaining service during upgrades and improving hardware efficiency as AI agents generate growing volumes of model requests.

Built for Kubernetes, llm-d coordinates model serving across computing infrastructure. IBM, its Red Hat subsidiary and Google LLC are its lead maintainers. They co-founded the project last year with CoreWeave Inc. and Nvidia Corp. Other contributors include Advanced Micro Devices Inc., Cisco Systems Inc., Hugging Face Inc., Intel Corp. and Mistral AI SAS.

The project offers organizations seeking more control over inference costs and…