Prompt Evaluation and Testing: From Vibes to Rigorous Evals, Part 1

Prompt changes need measurement; a harness makes the measurement repeatable so you ship improvements with confidence.

What AI does well here

• Build representative test sets (real traffic samples + edge cases + adversarial prompts)
• Define metrics appropriate to the task (correctness, faithfulness, format compliance, safety)
• Use LLM-as-judge for scalable evaluation, calibrated against human review
• Track per-version metrics so regressions are visible

What AI cannot do

• Substitute for human evaluation on the most important behaviors
• Catch behaviors not represented in the test set
• Replace production monitoring (test set evaluation is necessary, not sufficient)