By
Publication Date: 2026-03-17 00:22:00
Ashajari said infrastructure also needs to match modern GPU demands.
“For example, high-end graphics processing units like the NVIDIA H [Hopper] 100 require memory bandwidth (up to approximately 2 TB/s[terabytes] per second) and strong host throughput,” he said.
“Infrastructure must keep pace to avoid bottlenecks. Critically, organisations need to plan for the next memory bottleneck.”
CXL memory pooling provides ultra-low latency shared memory for larger models. This will be a key future requirement for on prem and edge AI as context windows expand.
Ashajari said this also ensured that memory architecture and bandwidth can handle AI training and inference workloads in GPU environments.
“AI performance lives or dies by how fast you can feed the GPUs,” he said. “This demands the removal of bottlenecks at every layer.
“Nutanix supports direct NVIDIA GPU acceleration on Acropolis Hypervisor (AHV) through passthrough and vGPU, providing workloads with access to GPU memory and bandwidth for AI and other compute-intensive tasks.”
By ensuring memory is allocated cleanly, data moves efficiently and the system stays balanced, even as models and datasets grow to maintain consistent AI performance.
Shadow AI impact
However, even with high-performance infrastructure, these organisations face challenges in how AI is deployed and governed across the business. This situation is compounded by the growth of shadow AI which also puts…