Prompt injection fundamentals: trust boundaries in agent systems

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-creators-prompt-injection-fundamentals

1. In the context of AI security, what is prompt injection most analogous to?

   1. Cross-site scripting, where malicious code runs in a user's browser
   2. Denial of service, where service availability is disrupted
   3. SQL injection, where untrusted input manipulates system behavior
   4. Buffer overflow, where memory boundaries are exceeded

2. Which statement best describes a trust boundary in an AI agent system?

   1. An encryption layer protecting data at rest
   2. A physical barrier preventing unauthorized hardware access
   3. The dividing line between trusted system prompts and untrusted external inputs
   4. A validation checkpoint that only runs on weekends

3. In the three-layer trust model presented, how is user input categorized?

   1. Untrusted, because users cannot be malicious
   2. Semi-trusted, because it requires validation before execution
   3. Fully trusted, because the user owns the system
   4. Ignored, because it is always overwritten by system prompts

4. According to the concepts covered, which capability is beyond what AI can currently achieve regarding prompt injection?

   1. Identifying repeated attack attempts
   2. Filtering known injection patterns
   3. Eliminating prompt injection risk entirely
   4. Detecting obvious malicious keywords in prompts

5. What is 'indirect injection' in the context of prompt injection attacks?

   1. When two system prompts conflict with each other
   2. When external data processed by the AI contains hidden malicious instructions
   3. When an AI model generates harmful output on its own
   4. When a user directly types malicious commands into a prompt

6. What is the primary purpose of tool isolation in agent systems?

   1. To simplify the user interface
   2. To prevent a compromised tool from accessing other system components
   3. To reduce the cost of running AI models
   4. To make the system run faster

7. What does the lesson say about relying on user confirmation to prevent prompt injection damage?

   1. User confirmation is the most effective defense
   2. Design for safety, not for blame transfer via confirmations
   3. User confirmation eliminates all liability
   4. Users should confirm every action the AI takes

8. When mapping trust boundaries for an agent, what should be specified for each tool?

   1. The number of lines in its documentation
   2. The programming language it was written in
   3. What input layer can trigger it and the validation gate before execution
   4. The tool's source code complexity

9. What type of content should be treated as untrusted in an AI agent system?

   1. Any external content reaching the model
   2. Only content stored in databases
   3. Only content from unknown IP addresses
   4. Only content typed directly by users

10. Why is output validation important in agent systems?

    1. To make the AI respond faster
    2. To improve the aesthetic quality of responses
    3. To ensure the AI doesn't generate harmful or unintended tool calls
    4. To reduce the cost of API calls

11. In the trust model, which layer should be considered fully trusted?

    1. System prompts
    2. Third-party plugin outputs
    3. External API responses
    4. User input

12. What should trigger validation gates in an agent system?

    1. Only malformed inputs
    2. Only inputs from new users
    3. Any input crossing a trust boundary
    4. Only inputs over 1000 characters

13. Why cannot AI completely solve the prompt injection problem?

    1. Because AI models are not powerful enough
    2. Because governments have banned AI security research
    3. Because distinguishing malicious instructions from legitimate requests is fundamentally difficult
    4. Because prompt injection doesn't really exist

14. What does the lesson recommend regarding tool-call validation rules?

    1. They should be changed daily
    2. They should have named owners responsible for them
    3. They should only be written for critical tools
    4. They should be created without any specific owner

15. In a prompt injection attack, what makes indirect injection particularly dangerous?

    1. It requires the user to type special characters
    2. It requires physical access to the server
    3. The malicious instructions come from a source the user trusts but cannot verify
    4. It only works on weekends