AI and Foster Care Risk Scoring: Allegheny's Lessons Generalized

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-ethics-safety-AI-and-foster-care-risk-scoring-r7a4-adults

1. A child-welfare agency deploys an AI screening tool without an appeal mechanism. Which ethical concern does this create?

   1. The algorithm will automatically improve its accuracy over time
   2. The tool will require less human oversight in decision-making
   3. The system functions as a star chamber, denying families the right to challenge the data used
   4. The agency will face increased operational costs from processing appeals

2. When a predictive risk score flags a family as high risk, what specific safeguard must be provided to that family?

   1. Immediate placement of children in foster care
   2. A cash voucher for family support services
   3. A mandatory meeting with a social worker within 48 hours
   4. Written notice explaining the score's role and a method to challenge specific data inputs

3. What does the Allegheny Family Screening Tool demonstrate about predictive scoring in child welfare?

   1. It demonstrates that poverty and neglect are easily distinguished by algorithms
   2. It illustrates how historical data can embed racial disparities into risk scores
   3. It shows that predictive tools require no external oversight
   4. It proves that AI can eliminate human bias entirely

4. According to the framework presented, what must be true for a child-welfare AI to operate ethically?

   1. All caseworkers must be replaced by the algorithm
   2. The tool must be developed by a government agency
   3. The system must achieve at least 95% accuracy
   4. Independent demographic audits must be conducted regularly

5. A predictive scoring system in child welfare shows high accuracy in lab testing but has never been audited for demographic disparities. What risk does this present?

   1. The score may embed historical bias from past discriminatory reporting
   2. Lab testing ensures the algorithm is ethical
   3. The tool will reduce costs automatically
   4. The system is guaranteed to be fair

6. What is a legitimate and appropriate function of AI in child-welfare hotline screening?

   1. Making final decisions about whether to remove children from homes
   2. Determining whether families are guilty of neglect without human review
   3. Predicting which families will become abusive in the future
   4. Aggregating referral history to help screeners triage incoming calls

7. An AI system is trained on ten years of child-welfare referral data. Which limitation is most likely to affect its fairness?

   1. The data is too old to be useful for modern cases
   2. The system will have difficulty reading handwritten case notes
   3. Older data contains too few cases to be statistically valid
   4. The historical data likely reflects over-reporting of Black and Indigenous families

8. What should an organization plan for before deploying any child-welfare predictive tool?

   1. A marketing campaign to inform families about the AI
   2. A celebration event to announce the new technology
   3. An independent audit to investigate bias and disparity metrics within 24 months
   4. A system shutdown after six months to evaluate results

9. When selecting a child-welfare AI vendor, what should be a key selection criterion?

   1. Whether the tool publishes its disparity metrics and audit results
   2. Whether the tool is the cheapest option available
   3. Whether the tool uses the most recent machine learning framework
   4. Whether the tool was developed by a university

10. A caseworker uses an AI risk score as the sole basis for deciding whether to investigate a family. Why is this problematic?

    1. The score is too expensive to question
    2. The score is always 100% accurate
    3. The score cannot distinguish between poverty signals and neglect signals
    4. The caseworker will be replaced by the algorithm

11. What does the lesson identify as a fundamental limitation of predictive scoring in child welfare?

    1. The technology requires too many caseworkers to operate
    2. The technology is too expensive for most agencies
    3. The technology cannot correct for over-reporting of marginalized communities
    4. The technology cannot process enough data quickly enough

12. A child-welfare agency implements a scoring tool that correctly identifies neglect in 85% of cases. Why might this still be ethically problematic?

    1. High overall accuracy can mask significant demographic disparities in error rates
    2. 85% accuracy means the tool is not useful
    3. The tool is not accurate enough for deployment
    4. The tool will increase the number of false negatives

13. What is the purpose of retrospective auditing for deployed child-welfare AI?

    1. To track outcomes and identify whether the tool produces disparate results across demographic groups
    2. To reduce the number of child welfare referrals
    3. To replace human caseworkers with automated systems
    4. To increase the speed of case processing

14. A new predictive tool claims it eliminates the need for human judgment in child-welfare decisions. Why is this claim problematic?

    1. Human-final-call is required because AI cannot distinguish poverty from neglect or operate ethically without oversight
    2. Human judgment is too slow for modern child welfare
    3. Human caseworkers are too expensive
    4. AI has been proven to be more reliable than human screeners

15. Why do families flagged as high risk need the ability to challenge specific data inputs used by the scoring system?

    1. Challenging data inputs improves the algorithm's accuracy for everyone
    2. Families must prove they are not guilty of neglect
    3. The law requires all data to be corrected before scoring can occur
    4. Incorrect or outdated data may produce unfair risk scores, and families must have a way to correct the record