AI-Assisted Document Review for Discovery: TAR 2.0 and Beyond

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-legal-AI-discovery-document-review-adults

1. When implementing LLM-based document review for a discovery matter, when should the review protocol be documented?

   1. During the midpoint of the review process to adjust for unexpected document volumes
   2. Only when responding to a court order or rule 26 challenge
   3. After the review is complete and before producing documents to opposing counsel
   4. Before deploying the AI system, as part of pre-review planning

2. What is the primary purpose of maintaining a control set of human-coded documents during AI-assisted review?

   1. To create a benchmark for billing the client for review hours
   2. To provide the AI system with training data to improve over time
   3. To validate the AI's accuracy by comparing machine decisions against human judgments
   4. To reduce the total number of documents requiring attorney review

3. According to the principles of defensible AI-assisted review, accuracy testing should be conducted:

   1. Only on documents flagged as potentially privileged
   2. Once at the beginning of the review to establish baseline metrics
   3. Only on the overall document population to get a single accuracy percentage
   4. Per responsiveness category to ensure the AI performs adequately on each document type

4. Why should privilege identification methodology be documented separately from general responsiveness methodology?

   1. Because privilege review is typically performed by different attorneys than responsiveness reviewers
   2. Because privilege identification typically requires more computational resources than responsiveness review
   3. Because privilege log production has different formatting requirements than document production
   4. Because courts apply heightened scrutiny to privilege determinations and need clear methodology documentation

5. Which of the following represents an inappropriate use of AI in document review?

   1. Using AI to categorize documents by document type for organizational purposes
   2. Allowing AI to make final determinations on whether documents contain privileged content
   3. Using AI to identify potential responsive documents for attorney approval
   4. Using AI to prioritize documents for attorney review based on responsiveness probability

6. On which type of document should AI not make the final responsiveness determination?

   1. Documents that are clearly responsive based on explicit search terms
   2. Documents where the responsiveness determination is a close call requiring legal judgment
   3. Documents that contain only metadata without substantive content
   4. Documents that are clearly non-responsive based on explicit search terms

7. Can AI-assisted review replace the meet-and-confer process with opposing counsel regarding TAR methodology?

   1. No, but AI can draft the initial meet-and-confer disclosure for attorney review
   2. No, because the meet-and-confer is a mandatory procedural requirement where methodology must be disclosed and agreed upon
   3. Yes, if the AI vendor provides documentation that satisfies disclosure requirements
   4. Yes, because AI systems are now sophisticated enough to address all methodological questions

8. When should defensibility documentation be created during an AI-assisted review?

   1. Contemporaneously with the review process, as the work is performed
   2. Only when the court requests it during a Rule 26 conference
   3. At the end of the matter as part of closing the file
   4. Retrospectively, after the review is complete, to support any future challenges

9. In designing a defensible AI review protocol, the seed-set methodology should specify:

   1. The criteria for selecting documents and the rationale for seed-set size
   2. Only the total number of documents in the seed set, not how they are selected
   3. The specific attorneys who will code the seed set documents
   4. How the AI will automatically expand the seed set during review

10. What role does statistical sampling play in QA during AI-assisted review?

    1. It replaces the need for any human review of AI-flagged documents
    2. It is used only at the very beginning of the review process
    3. It determines which documents should be prioritized for privilege review
    4. It provides a method to estimate overall accuracy without reviewing every document

11. A Rule 26 challenge to AI-assisted review methodology most likely focuses on:

    1. Whether the client was billed appropriately for review time
    2. Whether all documents were reviewed using AI rather than human reviewers
    3. Whether the AI vendor had sufficient insurance coverage
    4. Whether the review protocol was documented before deployment and whether validation was performed

12. When opposing counsel challenges your AI-assisted review methodology, which approach provides the strongest defense?

    1. Arguing that the burden of proof lies with the challenging party
    2. Explaining that the AI technology is proprietary and cannot be disclosed
    3. Presenting contemporaneous documentation of protocols, validation testing, and sampling methodology
    4. Claiming that AI-assisted review is now presumptively acceptable under federal rules

13. Which of the following tasks remains the responsibility of human attorneys even when using advanced AI systems?

    1. Categorizing documents by document type (email, memo, contract)
    2. Extracting dates and key data points from responsive documents
    3. Making final determinations that documents contain attorney-client privilege
    4. Identifying documents that are potentially responsive to discovery requests

14. AI systems should not make the final determination on documents that require:

    1. Extraction of structured data from forms
    2. Legal judgment about whether content is responsive to discovery requests
    3. Automatic classification into document type categories
    4. Simple keyword searches to identify relevant content

15. The meet-and-confer process in discovery cannot be replaced by:

    1. AI-generated disclosure documents for attorney review
    2. AI systems reviewing and agreeing to methodology
    3. Written correspondence between counsel
    4. A joint letter outlining agreed-upon methodology