Logging Agent Runs So You Can Debug Them Later

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-agentic-observability-r12a1-creators

1. Why is structured logging essential for debugging agent systems?

   1. It provides a replayable record of every decision and action the agent took
   2. It makes the agent execute tasks faster by caching results
   3. It reduces the cost of running agent infrastructure
   4. It automatically fixes bugs in the agent code

2. Which statement describes what AI CANNOT do with agent logs?

   1. AI can independently determine which step in a log was 'wrong'
   2. AI can emit a structured event for each tool call an agent makes
   3. AI can capture latency information for each tool execution
   4. AI can reconstruct an entire session from the event log alone

3. In the log entry structure {run_id, step_n, tool, input, output, status, latency_ms, model_version}, which field uniquely identifies a single agent execution session?

   1. run_id
   2. tool
   3. model_version
   4. step_n

4. Why must tool inputs in agent logs be treated with the same redaction rules as application logs?

   1. Agent logs are stored in a different format than application logs
   2. Redaction is required by law for all software logs
   3. Tool inputs may contain sensitive user data such as passwords, PII, or API keys
   4. Tool inputs are always encrypted by default

5. What is the relationship between a 'trace' and a 'log' in agent observability?

   1. A trace is a collection of related log entries showing the path of a single request
   2. They are interchangeable terms for the same thing
   3. A log contains trace data but trace does not contain log data
   4. Traces are for errors only, logs are for all events

6. If you want to debug an agent that failed three hours ago, what is the minimum requirement?

   1. You need to reproduce the exact conditions in real-time
   2. You need access to the original agent code
   3. You need to ask the user what went wrong
   4. You need a structured log that records each step with its inputs and outputs

7. Which field in the structured log entry would help identify performance bottlenecks?

   1. latency_ms
   2. status
   3. run_id
   4. model_version

8. A developer notices an agent produced incorrect output. Without structured logs, what can they actually debug?

   1. They can identify exactly which tool call caused the problem
   2. They cannot reconstruct the execution and must guess at the cause
   3. They can see which model version was used
   4. They can replay the execution to see what happened

9. What information is typically NOT captured in a structured agent log, even with comprehensive logging?

   1. Tool input parameters
   2. Internal AI reasoning that was never output
   3. Execution timing
   4. Tool output results

10. When should you include model_version in your agent logs?

    1. Only when the model produces an error
    2. Never, because it's not useful for debugging
    3. Always, because different model versions may behave differently
    4. Only when debugging machine learning issues

11. What is the primary advantage of structured logs over free-text logging?

    1. Structured logs don't require any storage
    2. Structured logs automatically fix bugs
    3. Structured logs are smaller in file size
    4. Structured logs can be programmatically queried and analyzed

12. Why is the 'status' field important in agent logging?

    1. It identifies the programming language used
    2. It tracks the monetary cost of the execution
    3. It determines which user can view the logs
    4. It indicates whether each tool call succeeded or failed

13. If you wanted to analyze the average performance of an agent over 1,000 runs, which log fields would be most useful?

    1. input and output only
    2. run_id and tool only
    3. model_version and step_n
    4. latency_ms and status

14. What distinguishes 'replay' from 'logging' in agent debugging?

    1. Replay requires the original agent to be running
    2. Replay is only for failed agent runs
    3. Logging and replay are the same thing
    4. Logging captures data; replay uses that data to reconstruct execution

15. A developer wants to find all tool calls that failed during a specific user's session. What is the most efficient approach?

    1. Filter logs by run_id and status fields
    2. Search the output field for error messages
    3. Read through all log files manually
    4. Check the model_version for errors