By Tanya Lenz
Publication Date: 2026-10-14 16:10:00
AI agents offer remarkable productivity gains in complex chip design workflows, but their tendency to generate plausible yet incorrect artifacts makes reliability a major barrier to production adoption. In multistage engineering workflows, an undetected error early in setup can silently propagate through downstream stages and final artifact generation. Evaluating and refining these agents against verified final outputs, rather than isolated intermediate steps, is essential for building trustworthy automation.
NVIDIA NeMo Gym, an open library for evaluating and improving models and agents using environments, addresses this challenge. The NeMo Gym agentic evaluation environment is designed to diagnose failure modes and track error propagation across complex, multistage workflows. By establishing a domain-specific verifier to inspect final outputs against verified reference artifacts, NeMo Gym enables engineers to trace complete rollout trajectories and identify root causes behind subtle errors.
This post walks through how NeMo Gym was used to evaluate and improve the reliability of the Siemens Fuse EDA AI Agent when calling characterization tools. The Siemens Fuse team used NeMo Gym to scale up evaluations and verify the resulting Liberty (.lib) files against known references and tracing—both common and rare failures through each rollout. Those findings guided changes to the agent’s skill and Model Context Protocol (MCP) tool descriptions, increasing the measured…
