How to Tell If Your Agent Run Was Actually Good

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-builders-agentic-ai-agent-eval-the-run-r10a8-teen

1. Which sentence best captures the main idea of 'How to Tell If Your Agent Run Was Actually Good'?

   1. Agents should always run without limits or oversight
   2. Tools and goals are unnecessary for agent design
   3. Score your agent on outcome, not on how clever the trace looked.
   4. Agents and chatbots are the same thing in every way

2. Which of the following is part of 'Some examples'?

   1. Hide tool calls from the operator
   2. Avoid taking any actions in the world
   3. Did the test suite end green
   4. Never log what the agent did

3. Which of the following is part of 'The rule of thumb'?

   1. a pretty trace that fails the task is still a failure.
   2. Approve all actions automatically
   3. Hide tool calls from the operator
   4. Ignore cost when scaling

4. Which of the following is part of 'You did it!'?

   1. Ignore cost when scaling
   2. Nice. You just practiced how to tell if your agent run was actually good. Do it three more times this week and it stops feeling like a trick and starts feeling like a tool.
   3. Approve all actions automatically
   4. Skip every form of evaluation

5. What is 'outcome metric' in this context?

   1. A reason to skip all logging
   2. A way to disable the agent's tools
   3. A core concept covered in How to Tell If Your Agent Run Was Actually Good
   4. A trick to bypass approvals

6. What is 'run quality' in this context?

   1. A trick to bypass approvals
   2. A way to disable the agent's tools
   3. A reason to skip all logging
   4. A core concept covered in How to Tell If Your Agent Run Was Actually Good

7. What is 'nudge count' in this context?

   1. A core concept covered in How to Tell If Your Agent Run Was Actually Good
   2. A reason to skip all logging
   3. A trick to bypass approvals
   4. A way to disable the agent's tools

8. What does evaluating 'the run' of an agent focus on?

   1. Just the final answer
   2. Only the prompt template
   3. Only the temperature setting
   4. The whole sequence of steps, tool calls, and recoveries — not only the final answer

9. An agent quietly retries a failed payment 50 times overnight. What design principle was missing?

   1. A bigger context window
   2. Bounded retries with human notification on repeated failure
   3. More creative prompting
   4. A larger model

10. Which is the clearest sign an 'agent' is really just a chatbot in disguise?

    1. It only produces text and never takes actions
    2. It uses a system prompt
    3. It can call a search tool
    4. It can remember last week's conversation

11. Which signal best tells you an agent is stuck in a runaway loop?

    1. It keeps repeating the same tool call with no new progress
    2. It finishes the task in one step
    3. It asks one clarifying question
    4. It returns a short summary and stops

12. What is the difference between an agent's memory and its context window?

    1. Context lasts forever; memory is cleared every minute
    2. Context is what the model sees right now; memory persists across runs
    3. Nothing — they are the same thing
    4. Memory is faster but less accurate than context

13. What is the best response when an agent suggests an action you do not understand?

    1. Run it twice to be sure
    2. Approve it to keep things moving
    3. Reject everything and stop using the agent
    4. Ask the agent to explain the action and its expected effect before approving

14. Why is keeping a human in the loop valuable for high-stakes agent actions?

    1. It removes the need for any logging
    2. It speeds the agent up
    3. It catches mistakes before they cause real-world harm
    4. It replaces the model entirely

15. Why is logging every tool call an agent makes a baseline requirement?

    1. Logs replace the need for testing
    2. Logs are needed to debug, audit, and explain agent behavior to users
    3. Logs make the model run faster
    4. Logs are only for legal teams