Validating AI agent output against a Zod or Pydantic schema

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-agentic-agent-output-schema-validation-creators

1. What is the fundamental difference between JSON-mode and schema validation?

   1. JSON-mode generates valid JSON syntax while schema validation checks both syntax and semantic structure
   2. JSON-mode is faster than schema validation in all cases
   3. Schema validation is only for Python, JSON-mode is for JavaScript
   4. JSON-mode guarantees the data is factually correct

2. Why is schema validation described as a 'safety net' in this context?

   1. It prevents the program from crashing
   2. It catches malformed or unexpected data before it reaches downstream systems
   3. It makes the AI run faster
   4. It automatically corrects errors in the data

3. When schema validation fails on an LLM response, what should happen next?

   1. Ignore the failure and continue processing
   2. Retry the request with the parser error message included
   3. Delete the response and ask for a new one
   4. Switch to a completely different AI model

4. How many total attempts does the lesson recommend before falling back to a safe default?

   1. 1 attempt total
   2. 2 attempts total
   3. 3 attempts total
   4. 5 attempts total

5. What information should be included when retrying after a parse failure?

   1. Only the original user prompt
   2. The original prompt plus the parser error details
   3. A completely new and different prompt
   4. Nothing specific is needed

6. What is the recommended action after exhausting all retry attempts?

   1. Continue retrying indefinitely until success
   2. Return a predefined safe default value or behavior
   3. Throw an unhandled exception to crash the system
   4. Request human intervention for every single failure

7. What aspect of a response can schema validation inherently NOT check?

   1. Whether the JSON syntax is valid
   2. The semantic correctness or factual truth of the content
   3. Whether all required fields are present
   4. The data types of field values

8. What is a 'hallucinated value that fits the schema'?

   1. A response with invalid JSON syntax
   2. A response that matches the schema structure but contains factually incorrect information
   3. A missing required field
   4. A field with an incorrect data type

9. What is the danger of using a bad schema combined with a 100% retry rate?

   1. The program will run too slowly
   2. The system will generate more accurate data over time
   3. Costs can escalate dramatically without improving results
   4. The AI model will automatically improve

10. What does 'defensive parsing' refer to in this context?

    1. Aggressively extracting data from any input possible
    2. Treating all LLM output as potentially malformed and validating it rigorously
    3. Avoiding parsing entirely for performance
    4. Only parsing responses from trusted AI models

11. What does 'structured output' mean when discussing LLM responses?

    1. Writing well-organized code comments
    2. LLM responses that conform to a defined data schema
    3. Using pretty-print JSON formatting
    4. Creating a nice command-line interface

12. Why can't AI validate semantic correctness of its own output?

    1. AI lacks JSON parsing capabilities
    2. AI doesn't have real-world grounding to verify facts against reality
    3. Schema validation tools don't support it
    4. It's computationally too expensive to check

13. If an LLM response passes schema validation, is it guaranteed to be accurate?

    1. Yes, always
    2. No—the data could still be factually wrong but structurally correct
    3. Yes, but only for numerical fields
    4. Yes, but only for text fields

14. Why is including parser error details when retrying valuable?

    1. It makes the error message longer for debugging
    2. It gives the model specific feedback about what went wrong, helping it adjust
    3. It's required by the JSON specification
    4. It prevents the model from making any errors

15. Why should LLM responses be treated as 'untrusted input'?

    1. LLMs always produce incorrect responses
    2. The model might generate unexpected or invalid data that requires validation before use
    3. Untrusted is the default security classification
    4. All AI output is malicious by default