AI Consolidating Scattered Runbooks Into One Source

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-operations-AI-and-runbook-consolidation-adults

1. An operations team discovers they have seven different runbooks for the same database failover procedure across Slack, Confluence, and a legacy wiki. Using AI to consolidate these would primarily help by:

   1. Automatically deleting the oldest versions to reduce clutter
   2. Identifying which version matches current production infrastructure
   3. Testing each runbook against a simulated incident to determine accuracy
   4. Clustering similar runbooks and drafting a merged version with conflict markers

2. After AI generates a merged runbook with several TODOs marking conflicts, what is the correct workflow before using it in production?

   1. Archive the merged runbook and return to using originals
   2. Delete the TODOs immediately since the AI marked them
   3. Keep the TODOs as-is so operators can choose during an incident
   4. Resolve each TODO by choosing one option and remove the alternatives

3. A merged runbook generated by AI includes a step referencing a tool that was deprecated six months ago. What should happen to this step?

   1. Replace it with the newest available tool automatically
   2. Mark it as a TODO requiring human review of current tools
   3. Delete it immediately without consultation
   4. Keep it as-is since the AI included it

4. Why should original runbooks be preserved even after AI produces a consolidated version?

   1. To maintain historical context that may be needed for audits
   2. In case the merged version fails during a real incident
   3. For compliance with data retention policies
   4. Because AI cannot generate truly complete documentation

5. Which task falls outside AI's capabilities in the runbook consolidation process?

   1. Clustering runbooks that cover the same incident type
   2. Identifying steps that differ across versions
   3. Drafting a merged document with TODO markers
   4. Determining which version reflects current production systems

6. An AI presents two runbook versions with conflicting restart commands: one says 'restart service' and another says 'restart server'. What should the AI mark with a TODO?

   1. Neither—AI should choose the safer option automatically
   2. The entire runbook since it contains errors
   3. Both statements as they represent different approaches
   4. The specific conflicting step showing both options

7. What does 'documentation hygiene' refer to in operations runbook management?

   1. Maintaining consistent and non-conflicting documentation across tools
   2. Storing documentation in encrypted formats
   3. The process of cleaning documentation formatting and grammar
   4. Ensuring all runbooks include timestamp metadata

8. An AI clusters runbooks and identifies one group containing versions from 2018, 2020, and 2024. The 2024 version has significantly fewer steps. What should an engineer prioritize?

   1. Deleting all versions except the most recent
   2. Using the longest version since it has more detail
   3. Verifying which steps in the 2024 version were removed and why
   4. Trusting the AI's clustering since it identified them as related

9. A team plans to delete their original runbooks after AI produces a merged version. This approach is problematic because:

   1. The merged version may contain errors not yet discovered
   2. Original runbooks contain proprietary information
   3. Storage costs are negligible so deletion is unnecessary
   4. AI-generated documents have no legal standing

10. An AI identifies two runbooks with identical step sequences but different tool names for the same action. This most likely indicates:

    1. One runbook was copied from the other with tool names changed
    2. The teams use different tools achieving the same result
    3. The runbooks are duplicates and one should be deleted
    4. The AI made an error in clustering

11. During a tabletop exercise, the merged runbook generated by AI fails to resolve an incident. What is the most likely root cause?

    1. The original runbooks were improperly archived
    2. The AI used deprecated language in the document
    3. The AI was not trained on enough examples
    4. The TODO conflicts were not resolved before the exercise

12. What distinguishes AI confidence from field-tested reliability in runbook consolidation?

    1. AI confidence is mathematical while reliability is subjective
    2. Field-tested reliability comes from actual incident use, not algorithmic certainty
    3. AI confidence applies to formatting while reliability applies to content
    4. There is no distinction—they are synonymous

13. An engineer owns a critical system and reviews the AI-merged runbook. Whose judgment ultimately determines if the runbook is ready for production use?

    1. The team's vote through majority decision
    2. The date of the most recent original runbook
    3. The AI system's confidence score
    4. The engineer's domain knowledge and system understanding

14. The AI generates a merged runbook but cannot determine which of three backup recovery procedures is currently correct. How should this be handled?

    1. Delete the step entirely since it's unclear
    2. Choose the first procedure alphabetically
    3. Mark the step as a TODO showing all three options
    4. Include all three procedures with clear conditional logic

15. A team has validated a merged runbook through a successful tabletop exercise. Can they now delete all original versions?

    1. No, originals must be kept forever for compliance
    2. Yes, but only if they are stored in cold storage
    3. Yes, validation confirms the merge is perfect
    4. No, originals should be retained until the runbook succeeds in a real incident