Introducing new Ray capabilities on SageMaker HyperPod | Amazon Web Services
Today, we are announcing new Ray capabilities on Amazon SageMaker HyperPod that integrate Ray with the HyperPod purpose-built infrastructure for…
Tag: HyperPod
Tiered KV cache for large LLMs on Amazon SageMaker HyperPod with Curvine | Amazon Web Services
Running large language model (LLM) inference at scale typically forces a KV cache trade-off: you either pay for oversized GPU…
Unlock efficient model deployment: Simplified Inference Operator setup on Amazon SageMaker HyperPod | Amazon Web Services
Amazon SageMaker HyperPod offers an end-to-end experience supporting the full lifecycle of AI development—from interactive experimentation and training to inference…
Scaling seismic foundation models on AWS: Distributed training with Amazon SageMaker HyperPod and expanding context windows | Amazon Web Services
This post is cowritten with Altay Sansal and Alejandro Valenciano from TGS. TGS, a geoscience data provider for the energy…
Accelerating AI model production at Hexagon with Amazon SageMaker HyperPod | Amazon Web Services
This blog post was co-authored with Johannes Maunz, Tobias Bösch Borgards, Aleksander Cisłak, and Bartłomiej Gralewicz from Hexagon. Hexagon is…
Manage Amazon SageMaker HyperPod clusters using the HyperPod CLI and SDK | Amazon Web Services
Training and deploying large AI models requires advanced distributed computing capabilities, but managing these distributed systems shouldn’t be complex for…
Checkpointless training on Amazon SageMaker HyperPod: Production-scale training with faster fault recovery | Amazon Web Services
Foundation model training has reached an inflection point where traditional checkpoint-based recovery methods are becoming a bottleneck to efficiency and…
HyperPod now supports Multi-Instance GPU to maximize GPU utilization for generative AI tasks | Amazon Web Services
We are excited to announce the general availability of GPU partitioning with Amazon SageMaker HyperPod, using NVIDIA Multi-Instance GPU (MIG). With…
HyperPod enhances ML infrastructure with security and storage | Amazon Web Services
Amazon SageMaker HyperPod is a purpose-built infrastructure for optimizing foundation model training and inference at scale. SageMaker HyperPod removes the…
Introducing auto scaling on Amazon SageMaker HyperPod | Amazon Web Services
Today, we’re excited to announce that Amazon SageMaker HyperPod now supports managed node automatic scaling with Karpenter, so you can…