Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models | NVIDIA Technical Blog

vm_admin

1 month ago

Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models | NVIDIA Technical Blog

By Moritz Reuss
Publication Date: 2026-06-15 12:00:00

Quick glossary for readers new to VLA/WAM terminology

VLA Vision-Language-Action model: a robot policy that starts from a pretrained VLM backbone and adapts it to generate actions from visual observations and language instructions. Large-scale VLM pretraining is a core part of the recipe. See Pi-0 and GR00T N1.

WAM World-Action Model: a policy that starts from a pretrained world-model or video backbone and adapts it to represent or predict how the scene changes over time and emit corresponding actions. We use WAM as the term throughout this post.

VLM Vision-Language Model: a model pretrained on image-text or video-text data to produce language outputs grounded in visual inputs, usually before being adapted for robot control.

Video backbone A pretrained video model reused as the central representation or generator inside a robot policy.

World model A model that predicts a future world state, conditioned on some action abstraction such as language, robot actions, or latent actions. The predicted state may be represented as images, video, point tracks, object states, or latent features. See the classic World Models paper and NVIDIA’s Cosmos world foundation model paper.

Grounding Connecting symbols (e.g. words in a language instruction) to the perceptual and motor referents that satisfy them. Language-to-action grounding in particular means turning an instruction like “pick up the red mug” into the visual percepts and motor commands that actually accomplish it. The…