By Patrick Moorhead
Publication Date: 2026-11-04 16:00:00
Turn production evidence into prioritized issues and custom evaluations that help you improve prompts, tools, models, memory, skills, and harness code.
When you build an agent for your domain, generic benchmark scores may not explain its failures. An agent might choose the wrong tool, follow the wrong workflow branch, miss relevant context, use an unnecessarily expensive model, or complete a task without meeting a domain-specific requirement.
Improving an agent requires more than model tuning. Prompts, tools, routing, memory, skills, control flow, and other code all shape production reliability. Traces help you investigate failures, while evaluations help you measure whether a change improves agent behavior.
This tutorial walks you through one complete optimization cycle: use NVIDIA NeMo Compass to identify a failure in recorded traces, create an evaluation with NVIDIA NeMo Eval Author, update the agent’s instructions, and compare results on development and held-out tasks. Follow along with the NVIDIA NemoClaw blueprint for agent optimization.
The blueprint uses Hermes Agent in an NVIDIA OpenShell sandbox, with recorded traces from a fictional company.
The agent harness is the optimization surface
One wrong tool call can derail an entire task. The cause may be an unclear tool description, a routing rule, or missing context in the agent harness. A trace helps identify where the run went wrong.
Our example agent is a Hermes enterprise assistant for a…
