Runbook Generation: Ops Memory That Survives Turnover

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-operations-runbook-generation-adults

1. A runbook written today is estimated to retain what percentage of accuracy one month later, according to the concepts covered?

   1. 80%
   2. 20%
   3. 95%
   4. 50%

2. Which statement best describes what AI can contribute to runbook creation?

   1. AI can transform structured incident data into runbook drafts for human refinement
   2. AI can predict future incidents with sufficient accuracy to prevent them entirely
   3. AI can automatically execute runbook steps during incidents without human oversight
   4. AI can generate accurate runbooks from scratch based on system documentation

3. Which components should be captured as structured incident timeline data?

   1. CPU metrics, memory usage, and network latency
   2. Alert, action, observation, and outcome
   3. Only the final resolution and root cause
   4. Customer complaints and business impact

4. What does 'drift' refer to in the context of AI-generated runbooks?

   1. Version control conflicts between multiple authors
   2. The rate at which runbook formatting becomes outdated
   3. A measurable difference between predicted and actual resolution paths indicating system changes
   4. The gradual degradation of server hardware over time

5. What must always accompany an AI-generated runbook entry before it is considered complete?

   1. A timestamp from the incident management system
   2. Human editor review and refinement
   3. Automated testing validation
   4. Sign-off from the security team

6. According to the concepts presented, why should AI-generated runbooks never auto-run destructive commands?

   1. Because auto-running any commands violates change management policies
   2. Because they consume too much computational resources
   3. Because the runbook is a guide for a human responder, not a script to execute
   4. Because destructive commands require root access that AI cannot obtain

7. What is the purpose of cross-linking related incidents when maintaining runbooks?

   1. To ensure compliance with audit requirements
   2. So patterns emerge across incidents
   3. To assign blame to specific team members
   4. To reduce the total number of documentation pages

8. A runbook entry should include which of the following elements?

   1. Only resolution steps and commands to run
   2. Contact information for all stakeholders
   3. Symptoms, diagnostic checks, causes, resolution steps, and escalation criteria
   4. The entire incident timeline from start to finish

9. What type of inference should be marked in a generated runbook entry?

   1. Information confirmed by multiple team members
   2. Standard Linux commands that are widely known
   3. Basic troubleshooting steps that apply to any system
   4. Anything inferred but not confirmed from the incident timeline

10. What does drift detection measure in an operational context?

    1. User behavior changes in the application
    2. The difference between documented and actual resolution approaches over time
    3. Memory usage patterns on production servers
    4. Network latency between data centers

11. What is the primary reason runbooks become inaccurate over time?

    1. The system being run changes while the documented steps remain static
    2. New engineers don't read them
    3. Engineers forget to update them
    4. The documentation format becomes obsolete

12. Why is it important to include rollback notes in resolution steps?

    1. To document who made changes for accountability
    2. To enable reverting changes if the resolution causes further issues
    3. To provide historical context for future incidents
    4. To satisfy compliance auditors

13. What relationship does the lesson describe between incidents and runbooks?

    1. Incidents are problems caused by inadequate runbooks
    2. Runbooks are downstream artifacts that should be derived from incident data
    3. Incidents and runbooks are unrelated documentation streams
    4. Runbooks are inputs that generate incidents

14. What should a responder do when using an AI-generated runbook draft during an incident?

    1. Execute it exactly as written without modification
    2. Treat it as a starting point requiring human judgment and editing
    3. Forward it to management for approval
    4. Discard it and write a new runbook from scratch

15. The lesson notes that fast-changing details like product names, prices, and policies should be treated as what when using generated runbooks?

    1. Examples to verify before use
    2. Fixed content that doesn't require verification
    3. Added to the dependency graph
    4. Removed from all runbooks