Catastrophic Risk, Without the Panic

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-safety-catastrophic-risk-intro-builders

1. What is the core idea behind "Catastrophic Risk, Without the Panic"?

   1. Measured people at serious labs and universities publicly worry about AI going very wrong. Here is what they mean, what they disagree about, and how to read the headlines.
   2. Use PPO (or another RL algorithm) to fine-tune the policy against the reward mod…
   3. mesa-optimization
   4. proxy goal

2. Which term best describes a foundational idea in "Catastrophic Risk, Without the Panic"?

   1. existential risk
   2. catastrophic risk
   3. CBRN
   4. model autonomy

3. A learner studying Catastrophic Risk, Without the Panic would need to understand which concept?

   1. catastrophic risk
   2. CBRN
   3. existential risk
   4. model autonomy

4. Which of these is directly relevant to Catastrophic Risk, Without the Panic?

   1. catastrophic risk
   2. existential risk
   3. model autonomy
   4. CBRN

5. Which of the following is a key point about Catastrophic Risk, Without the Panic?

   1. CBRN: chemical, biological, radiological, nuclear. Can a model meaningfully help a novice build a we…
   2. Cyber: can a model conduct offensive cyber operations above a specific skill bar?
   3. Model autonomy: can a model operate independently over long horizons, make copies of itself, acquire…
   4. Persuasion and manipulation: can a model influence beliefs at scale in ways that destabilize institu…

6. Which of these does NOT belong in a discussion of Catastrophic Risk, Without the Panic?

   1. Cyber: can a model conduct offensive cyber operations above a specific skill bar?
   2. CBRN: chemical, biological, radiological, nuclear. Can a model meaningfully help a novice build a we…
   3. Model autonomy: can a model operate independently over long horizons, make copies of itself, acquire…
   4. Use PPO (or another RL algorithm) to fine-tune the policy against the reward mod…

7. Which statement is accurate regarding Catastrophic Risk, Without the Panic?

   1. Cyber: frontier models complete apprentice-level offensive security tasks 50% of the time (UK AISI 2…
   2. Autonomy: METR's time-horizon benchmark shows AI-completable task length doubling every 4-7 months.
   3. CBRN uplift: measurable but limited. 2024-2025 evaluations show models can accelerate research tasks…
   4. Persuasion: hard to measure. Some lab evals were tracked then dropped as benchmarks matured.

8. Which of these does NOT belong in a discussion of Catastrophic Risk, Without the Panic?

   1. CBRN uplift: measurable but limited. 2024-2025 evaluations show models can accelerate research tasks…
   2. Autonomy: METR's time-horizon benchmark shows AI-completable task length doubling every 4-7 months.
   3. Use PPO (or another RL algorithm) to fine-tune the policy against the reward mod…
   4. Cyber: frontier models complete apprentice-level offensive security tasks 50% of the time (UK AISI 2…

9. What is the key insight about "What the 2023 statement actually said" in the context of Catastrophic Risk, Without the Panic?

   1. The famous one-sentence Center for AI Safety statement signed by Hinton, Bengio, Altman, Hassabis and hundreds of resear…
   2. Use PPO (or another RL algorithm) to fine-tune the policy against the reward mod…
   3. mesa-optimization
   4. proxy goal

10. What is the key insight about "Two takes you should distrust equally" in the context of Catastrophic Risk, Without the Panic?

    1. Use PPO (or another RL algorithm) to fine-tune the policy against the reward mod…
    2. Anyone confidently saying extinction is coming in X years is making a claim with no track record of accurate long-range …
    3. mesa-optimization
    4. proxy goal

11. What is the recommended tip about "Key insight" in the context of Catastrophic Risk, Without the Panic?

    1. Use PPO (or another RL algorithm) to fine-tune the policy against the reward mod…
    2. mesa-optimization
    3. Measured people at serious labs and universities publicly worry about AI going very wrong.
    4. proxy goal

12. Which statement accurately describes an aspect of Catastrophic Risk, Without the Panic?

    1. Use PPO (or another RL algorithm) to fine-tune the policy against the reward mod…
    2. mesa-optimization
    3. proxy goal
    4. You have seen the headlines: AI could end humanity. You have also seen the opposite: AI doom is sci-fi nonsense.

13. What does working with Catastrophic Risk, Without the Panic typically involve?

    1. OpenAI's Preparedness Framework and Anthropic's Responsible Scaling Policy both enumerate similar categories for high-severity risk.
    2. Use PPO (or another RL algorithm) to fine-tune the policy against the reward mod…
    3. mesa-optimization
    4. proxy goal

14. Which of the following is true about Catastrophic Risk, Without the Panic?

    1. Use PPO (or another RL algorithm) to fine-tune the policy against the reward mod…
    2. The big idea: catastrophic risk is a real research agenda with real evidence.
    3. mesa-optimization
    4. proxy goal

15. Which best describes the scope of "Catastrophic Risk, Without the Panic"?

    1. It is unrelated to ethics workflows
    2. It applies only to the opposite beginner tier
    3. It focuses on Measured people at serious labs and universities publicly worry about AI going very wrong. Here is w
    4. It was deprecated in 2024 and no longer relevant