Build real-time voice applications with vLLM-Omni on SageMaker AI – Part 1 | Amazon Web Services

Voice agents, interactive learning applications, accessibility tools, and customer service assistants need to respond without long silent pauses. In this tutorial, you deploy a text-to-speech (TTS) model on Amazon SageMaker AI that can start playing speech before it finishes generating the full response. You use the AWS vLLM-Omni Deep Learning Container (DLC) to deploy Qwen3-TTS, stream text in and audio out over one persistent bidirectional connection, and try the workflow through a Gradio application.

AWS Deep Learning Containers provide Docker images with deep learning frameworks and dependencies for training and inference on AWS. AWS provides deployment guidance for broadly adopted serving frameworks such as vLLM and SGLang. This post is Part 1 of a series about specialized DLCs, including vLLM-Omni, WhisperX, and llama.cpp. It focuses on streamed speech for real-time voice applications. Part 2 applies the vLLM-Omni DLC to image and video generation. The series…