Validate

The validate command checks evaluation files for schema errors without running them.

Usage

agentv validate evals/my-eval.yaml

Validate multiple files:

agentv validate evals/**/*.yaml

What It Checks

• YAML/JSONL syntax
• Required fields: id, input, and at least one of criteria, expected_output, assertions, or turns
• Grader references (command paths, prompt files)
• Target references match entries in targets.yaml
• Rubric structure and field types

When to Use

• Before running evaluations to catch config errors early
• In CI/CD pipelines as a pre-check
• After editing eval files to verify correctness

agentv validate replaces the old eval mock dry-run use case for schema and configuration checks. It does not execute targets and does not produce quality scores. When you need no-live-LLM quality validation, run against an oracle/reference target or use frozen transcript/replay fixtures so graders see real candidate output.