Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use

TL;DR

This study explores how power dynamics within public sector agencies influence decisions on AI adoption, highlighting barriers to stakeholder involvement and suggesting practical support for more participatory AI practices.

Contribution

It introduces an anthropological 'study up' approach to analyze decision-making processes of agency leaders regarding AI implementation.

Findings

Power relations shape AI decision-making processes.

Barriers limit stakeholder involvement in AI choices.

Agency leaders seek support for inclusive AI decision practices.

Abstract

As public sector agencies rapidly introduce new AI tools in high-stakes domains like social services, it becomes critical to understand how decisions to adopt these tools are made in practice. We borrow from the anthropological practice to ``study up'' those in positions of power, and reorient our study of public sector AI around those who have the power and responsibility to make decisions about the role that AI tools will play in their agency. Through semi-structured interviews and design activities with 16 agency decision-makers, we examine how decisions about AI design and adoption are influenced by their interactions with and assumptions about other actors within these agencies (e.g., frontline workers and agency leaders), as well as those above (legal systems and contracted companies), and below (impacted communities). By centering these networks of power relations, our findings shed light on how infrastructural, legal, and social factors create barriers and disincentives to the involvement of a broader range of stakeholders in decisions about AI design and adoption. Agency decision-makers desired more practical support for stakeholder involvement around public sector AI to help overcome the knowledge and power differentials they perceived between them and other stakeholders (e.g., frontline workers and impacted community members). Building on these findings, we discuss implications for future research and policy around actualizing participatory AI approaches in public sector contexts.