Microsoft Azure CTO Mark Russinovich discussed how Microsoft is expanding its cloud infrastructure to meet the growing demand for AI at the Microsoft Build 2024 conference. The company has scaled its AI infrastructure 30 times since November 2023 to support large language models (LLM) with increased efficiency being a key focus. The rapid growth in model sizes has led to a significant increase in GPU performance, memory capacity, and power consumption, presenting challenges in cooling systems. Microsoft has introduced liquid refrigeration, such as the Maya racks, as part of its new design for cloud data centers. In addition to cooling, the company is also optimizing power oversubscription and utilizing high-speed networking technologies like InfiniBand interconnect to enhance scale and performance.
Microsoft has developed Project Forge, an internal AI workload platform that treats GPUs as a shared resource pool to improve resource allocation and utilization. This approach has resulted in over 95% utilization of infrastructure resources for Microsoft’s own training purposes. Overall, Microsoft’s innovations in AI infrastructure reflect a commitment to meeting the growing demands for AI in a scalable and efficient manner.
This article provides insights into Microsoft’s efforts to build a comprehensive AI infrastructure to support the increasing demand for AI technologies. By focusing on efficiency, scalability, and innovation, Microsoft is positioning itself to meet the challenges posed by large language models and the demand for AI across various industries. Mark Russinovich’s presentation at the Microsoft Build 2024 conference highlights the company’s commitment to staying at the forefront of cloud AI infrastructure development. With advancements in cooling, power optimization, networking, and resource allocation, Microsoft is addressing key challenges in scaling AI infrastructure to meet the demands of the future.
Article Source
https://www.itprotoday.com/cloud-computing-and-edge-computing/microsoft-details-how-its-building-ai-cloud-infrastructure