Release evidence guide
How to test an AI agent before giving it more access
Test the agent on the work it will actually do. Include routine tasks, exceptions, denied actions, missing information, old records, and hostile inputs. Check both the result and whether it followed the rules. Grant more access only after an explicit release decision, with a way to undo it.
- Good fit
- Use workflow-specific evaluation whenever an agent reads private context, calls tools, recommends consequential actions, or moves from drafting toward external execution.
- Pause when
- A single demonstration, aggregate accuracy score, or vendor benchmark cannot establish readiness when failures have different costs or when the agent’s permissions differ from the test environment.
When this approach helps
- Stakeholders describe the agent as working but cannot name the cases it passed.
- A new tool, prompt, model, policy, or data source can change behavior after launch.
- The team needs a repeatable reason to expand, hold, or reduce the agent’s authority.
Follow these steps
- 01
Define the possible results
Name successful, needs-human, blocked, missing-evidence, tool-failure, and unknown-action outcomes. Check answer quality and whether the agent stopped when it should.
- 02
Build a reusable set of cases
Use representative patterns with privacy protections. Keep difficult exceptions. Save versions of inputs and expected results so changes in models, instructions, rules, and tools can be compared.
- 03
Hide the answers during the test
Keep expected results hidden from the agent. Use repeatable automated checks where possible. Have qualified reviewers examine disputed judgments, tool calls, and stopping behavior.
- 04
Approve the release and keep checking
Record allowed actions, required scores, unresolved risks, the approver, and recovery steps. Keep checking live results: passing test cases cannot guarantee future behavior.
Documents and records to keep
Versioned evaluation set
Fixtures identify source pattern, expected outcome class, required evidence, prohibited actions, privacy treatment, and why the case matters.
Promotion decision
A signed record states the tested versions, scores by case class, known misses, granted authority, monitoring plan, owner, and rollback trigger.
Regression report
Each material system change is compared against prior behavior, with new failures routed to a named owner before the release can expand authority.
Common questions
How many cases are enough for an AI agent evaluation?
No fixed count proves readiness. Cover every meaningful outcome and risk class, then add cases until new samples stop revealing major behavior gaps. High-consequence or diverse workflows require broader evidence than a narrow drafting task.
Should AI agent evaluations use production data?
Use representative patterns while applying privacy, access, and retention rules. De-identification or synthetic fixtures may be appropriate, but verify that they preserve the edge conditions and source relationships the agent will actually encounter.
What should happen when an agent fails one rare case?
Assess the consequence and whether the failure reveals a general boundary weakness. Keep or reduce authority, add the case to regression coverage, fix the relevant data, policy, tool, or judgment layer, and require a new promotion decision.