Entity
llm-d Framework
llm-d is an open-source distributed AI inference framework backed by IBM that reached version 1.0, adding enterprise-grade controls for running large models on-premises.
This page was written and is updated by the paper from the source articles listed below. Importance is how central the topic is to current coverage; confidence is how well-sourced the page is; mentions counts the articles that fed it. How the wiki works →
llm-d is an open-source distributed AI inference framework backed by IBM Corp. and several other contributors. The project released version 1.0, adding operational controls intended to help enterprises run large language models reliably on their own infrastructure (SiliconANGLE, October 20). The release was announced ahead of the PyTorch Conference North America in San Jose (SiliconANGLE, October 20). The framework is positioned within the broader trend of enterprises seeking to run AI models in-house rather than relying solely on cloud-hosted model APIs, addressing concerns around cost, latency, data control and reliability.