Prompt-Injection Risks Specific To ChatGPT Plugins And Connectors

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-openai-prompt-injection-risks-creators

1. Why is the model fundamentally unable to reliably distinguish instructions from data?

   1. The model lacks sufficient training on security concepts
   2. The model prioritizes user requests over system prompts
   3. The model's context window is too small to track instruction origins
   4. The model processes all text as tokens without inherent markers separating commands from content

2. A user enables a browser tool that lets ChatGPT visit websites. What is the primary injection risk?

   1. Webpages can cause the model to generate incorrect factual answers
   2. The model might download malware that executes on the user's computer
   3. Webpages can contain hidden text designed to instruct the agent to take unauthorized actions
   4. The browser tool exposes the user's browsing history to the model

3. What is the worst-case outcome if an attacker successfully injects instructions via an email connector?

   1. The email account becomes locked due to failed login attempts
   2. Sensitive emails are forwarded to the attacker without the user's knowledge
   3. The model deletes all emails in the inbox
   4. The model sends spam to all contacts in the address book

4. What specific attack was demonstrated by security researchers involving a poisoned document in a connected drive?

   1. The document caused ChatGPT to crash and lose conversation history
   2. The document created a backdoor account in the user's cloud storage
   3. The document instructed the model to search for credentials and exfiltrate them via a markdown image URL
   4. The document triggered an automatic software update that disabled security features

5. What is the recommended defense against malicious content in document Q&A tools?

   1. Disable document Q&A entirely and use manual search instead
   2. Use only PDF documents, never Word files
   3. Strip or sanitize untrusted documents before indexing them for search
   4. Require the model to confirm each sentence in a document before answering questions

6. Why are calendar invites considered a prompt injection risk?

   1. The model cannot accurately parse calendar time formats
   2. Calendar invite descriptions are user-controlled and reach the agent with potential instructions
   3. Calendar invites use a proprietary format that confuses language models
   4. Calendar APIs have known vulnerabilities that attackers exploit

7. What is an approval gate in the context of ChatGPT tool use?

   1. A mandatory pause where the model must explicitly get human permission before executing sensitive actions
   2. A technical barrier that prevents the model from accessing certain data
   3. A setting that automatically approves all requests from verified sources
   4. A password required to enable any connector

8. What does 'least privilege' mean when applied to ChatGPT connectors?

   1. Limiting the number of queries a connector can process per day
   2. Requiring users to prove they are at least 18 years old
   3. Giving each connector only the minimum permissions needed for its current task
   4. Only granting temporary access to premium features

9. What is the recommended practice for 'send' or 'write' actions in connected ChatGPT setups?

   1. Allow the model to send automatically but log for review later
   2. Require explicit human approval before any send/write action executes
   3. Never allow send/write actions under any circumstances
   4. Only allow sending to addresses already in the contact list

10. What is the purpose of conducting a 60-day audit of active connectors and Custom GPT actions?

    1. To reset the model's memory and clear potential injection artifacts
    2. To receive new feature updates from OpenAI
    3. To comply with legal requirements for data retention
    4. To verify that permissions haven't expanded and that active tools remain necessary

11. The lesson compares treating retrieved content to how a good editor treats a press release. What does this mean in practice?

    1. The content requires legal review before the model can use it
    2. The content should always be verified by a second human editor
    3. The content should be published immediately without modification
    4. The content is useful as input but should never be treated as instructions to follow

12. If you paste a webpage into a ChatGPT session that has connectors enabled, what risk are you creating?

    1. The webpage will be shared with other ChatGPT users
    2. You import potentially adversarial instructions into a context where the model can act
    3. The webpage will slow down the model's response time
    4. The model will remember the webpage forever

13. What is the community consensus among security researchers regarding prompt injection in connected ChatGPT setups?

    1. Prompt injection has been completely solved by current AI models
    2. Prompt injection is the dominant practical risk and no model-side defense is sufficient on its own
    3. Prompt injection only affects enterprise accounts, not individual users
    4. Prompt injection risks are negligible compared to traditional hacking

14. What warning sign indicates a possible prompt injection attack in progress?

    1. The model asks for clarification more frequently
    2. The agent suddenly wants to take an unexpected action like emailing someone it hasn't mentioned before
    3. The model begins responses with more hedged language
    4. The model responds more slowly than usual

15. What is the purpose of logging everything an agent does?

    1. To train future AI models on user behavior
    2. To improve the model's language capabilities
    3. To charge the user for API usage
    4. To create an audit trail that serves as the primary forensic tool if something goes wrong