Red-Team Evals

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-creators-red-team-evals

1. What is the core idea behind "Red-Team Evals"?

   1. Benchmarks measure what you ask. Red-teaming measures what breaks. Learn to test for failure modes, not capabilities. For AI, red teams probe for harmful outputs, jailbreaks, bias, leakage of training data, and dangerous capabilities.
   2. confidence interval
   3. peer review
   4. generalization

2. Which term best describes a foundational idea in "Red-Team Evals"?

   1. jailbreak
   2. red team
   3. prompt injection
   4. threat model

3. A learner studying Red-Team Evals would need to understand which concept?

   1. red team
   2. prompt injection
   3. jailbreak
   4. threat model

4. Which of these is directly relevant to Red-Team Evals?

   1. red team
   2. jailbreak
   3. threat model
   4. prompt injection

5. Which of the following is a key point about Red-Team Evals?

   1. Content harms: toxicity, illegal instructions, CSAM refusals
   2. Jailbreaks: prompts that bypass safety guidelines
   3. Privacy leaks: reproducing training data or PII
   4. Prompt injection: external content overrides system instructions

6. Which of these does NOT belong in a discussion of Red-Team Evals?

   1. Content harms: toxicity, illegal instructions, CSAM refusals
   2. confidence interval
   3. Jailbreaks: prompts that bypass safety guidelines
   4. Privacy leaks: reproducing training data or PII

7. Which statement is accurate regarding Red-Team Evals?

   1. Generate adversarial prompts per threat
   2. Run them through the model
   3. Write a threat model: who might attack and what they want
   4. Score responses by a harms rubric

8. Which of these does NOT belong in a discussion of Red-Team Evals?

   1. confidence interval
   2. Write a threat model: who might attack and what they want
   3. Generate adversarial prompts per threat
   4. Run them through the model

9. What is the key insight about "Automated red-teaming" in the context of Red-Team Evals?

   1. Tools like PAIR, TAP, and DAN can generate jailbreak prompts automatically.
   2. confidence interval
   3. peer review
   4. generalization

10. What is the key insight about "Red-team findings are dual-use" in the context of Red-Team Evals?

    1. confidence interval
    2. Publishing specific jailbreaks can help attackers more than defenders.
    3. peer review
    4. generalization

11. What is the recommended tip about "Ground your practice in fundamentals" in the context of Red-Team Evals?

    1. confidence interval
    2. peer review
    3. Every AI capability has an underlying mechanism. Understanding that mechanism tells you where it'll fail — which is more…
    4. generalization

12. Which statement accurately describes an aspect of Red-Team Evals?

    1. confidence interval
    2. peer review
    3. generalization
    4. Red-teaming means deliberately trying to break a system. For AI, red teams probe for harmful outputs, jailbreaks, bias, leakage of training …

13. What does working with Red-Team Evals typically involve?

    1. The big idea: capability evals ask 'can it?' Red-team evals ask 'what happens when someone tries to break it?' You need both.
    2. confidence interval
    3. peer review
    4. generalization

14. Which best describes the scope of "Red-Team Evals"?

    1. It is unrelated to foundations workflows
    2. It focuses on Benchmarks measure what you ask. Red-teaming measures what breaks. Learn to test for failure modes,
    3. It applies only to the opposite beginner tier
    4. It was deprecated in 2024 and no longer relevant

15. Which section heading best belongs in a lesson about Red-Team Evals?

    1. confidence interval
    2. peer review
    3. Categories of red-team probes
    4. generalization