Algorithmic Harms in Child Welfare: Uncertainties in Practice, Organization, and Street-level Decision-Making

TL;DR

This paper examines how algorithmic systems in child welfare agencies influence decision-making, highlighting issues of fairness, process disruption, and the need for organizational investment to improve outcomes.

Contribution

It provides an ethnographic analysis of algorithm integration in child welfare, revealing practical challenges and proposing a trauma-informed algorithmic tool for better decision-making.

Findings

Algorithmic systems can cause process-oriented harms and decision inconsistencies.

Caseworkers perform additional repair work due to functionality issues.

A trauma-informed algorithmic tool improves outcomes with organizational investment.

Abstract

Algorithms in public services such as child welfare, criminal justice, and education are increasingly being used to make high-stakes decisions about human lives. Drawing upon findings from a two-year ethnography conducted at a child welfare agency, we highlight how algorithmic systems are embedded within a complex decision-making ecosystem at critical points of the child welfare process. Caseworkers interact with algorithms in their daily lives where they must collect information about families and feed it to algorithms to make critical decisions. We show how the interplay between systemic mechanics and algorithmic decision-making can adversely impact the fairness of the decision-making process itself. We show how functionality issues in algorithmic systems can lead to process-oriented harms where they adversely affect the nature of professional practice, and administration at the agency, and lead to inconsistent and unreliable decisions at the street level. In addition, caseworkers are compelled to undertake additional labor in the form of repair work to restore disrupted administrative processes and decision-making, all while facing organizational pressures and time and resource constraints. Finally, we share the case study of a simple algorithmic tool that centers caseworkers' decision-making within a trauma-informed framework and leads to better outcomes, however, required a significant amount of investments on the agency's part in creating the ecosystem for its proper use.