Skip to content
New this week26 prompts for Sora, Veo and Runway video workflows.
Explore video prompts→

Search AIPrompt123

Search prompts, categories, platforms and repositories

Build an evaluation set for an agent

This copy-ready agents prompt helps you build an evaluation set for an agent with fewer blank-page decisions. Paste it into Claude and ChatGPT, fill the variables, and you get a structured first draft you can edit. It is written for repeatable results and tells you what to check before publishing.

4.8(45 ratings)143 copies
Updated By Author
46 words · v1.2

Fill in the blanks · 1 variable

Claude promptReplace every highlighted token before sending
<task>
Create an evaluation set for an agent that {purpose}. Include 20 test cases across happy path, ambiguous input, missing data, adversarial input, tool failure and policy edge cases. For each give the input, expected behaviour and pass criteria. Add a scoring rubric and a regression process.
</task>

Think carefully before answering. Keep any reasoning separate and put the final deliverable inside <answer> tags.
Open in Claude

Comments

Share what worked, what broke and which model you used. Comments appear after posting and are moderated for spam.

Saved with an anonymous browser ID. No login needed.

How to use this prompt

  1. 1

    Open Claude and start a new chat. Paste the prompt exactly as written.

  2. 2

    Replace each placeholder ({purpose}) with your own details. Leave the instructions and the output format untouched.

  3. 3

    Add one concrete example or data point if the model's first answer is vague. Specific input produces specific output.

  4. 4

    Run it once, then ask for a single revision with a specific critique rather than regenerating from scratch.

  5. 5

    Review the result. Tool permissions, spend limits, approval gates, rollback and the eval results before launch.

Worked example

This is the kind of input the prompt expects and the shape of output it produces. Your result will differ with the model and the details you supply.

Example input

Purpose: triaging support tickets

Example output

**Agent:** reads the request, checks the allowlist, drafts an action and pauses for approval before any write. **Stop conditions:** missing required fields, ambiguous target, two consecutive tool errors, or any request touching billing or deletion. **Eval cases:** valid update, wrong customer ID, prompt injection in a document, tool timeout, duplicate submission and a request to bypass approval. **Failure found in review:** the agent could retry a payment twice. Fix: idempotency key plus a hard stop before the second attempt.

Tips for a better result

  • Design the stop conditions before the happy path; most incidents live in the loop.
  • Give each tool one job and make errors actionable for the model.
  • Require human confirmation for irreversible actions, no exceptions.
  • Change one variable at a time and compare the outputs; it is the fastest way to learn what this model responds to.

Common mistakes

  • Giving an agent broad write access without approval gates or spend limits.
  • Trusting retrieved documents as instructions rather than data.
  • Shipping without tracing, so a failed run cannot be reconstructed.
Verified Knowledge · FAQ

Questions about this prompt

Practical answers about running the build an evaluation set for an agent prompt.

Related prompts

Same job, a different angle — or the next step in the workflow.

Design a structured extraction pipeline

This copy-ready agents prompt helps you design a structured extraction pipeline with fewer blank-page decisions. Paste it into ChatGPT and Claude, fill the variables, and you get a structured first draft you can edit. It is written for repeatable results and tells you what to check before publishing.

ChatGPTClaudeDeepSeek
4.7(56)158 copies

Write an agent system prompt with stop conditions

This copy-ready agents prompt helps you write an agent system prompt with stop conditions with fewer blank-page decisions. Paste it into ChatGPT and Claude, fill the variables, and you get a structured first draft you can edit. It is written for repeatable results and tells you what to check before publishing.

ChatGPTClaudeGemini
4.7(59)186 copies

Design a human-in-the-loop approval step

Design a human-in-the-loop approval step with a prompt built for Claude and ChatGPT. It supplies context, constraints and an output contract, then asks the model to surface assumptions and gaps. Copy it, fill the variables and treat the result as a structured draft, not a final answer.

ClaudeChatGPT
4.8(33)222 copies

Written and reviewed by Author

AIPrompt123 Contributor

Contributing author and prompt engineer at AIPrompt123.

Last reviewed 15 November 2026 · version 1.2 · tested on Claude, ChatGPT