Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making

TL;DR

This paper explores the challenges and opportunities in designing algorithmic tools that support fairness and accountability in high-stakes public sector decisions, based on interviews with practitioners across multiple countries.

Contribution

It identifies gaps between current research and practical needs in public sector ML applications, proposing design opportunities for transparency and domain knowledge integration.

Findings

Practitioners face organizational and institutional constraints.

Current research often overlooks practical realities.

Design opportunities include tracking concept drift and building transparency tools.

Abstract

Calls for heightened consideration of fairness and accountability in algorithmically-informed public decisions---like taxation, justice, and child protection---are now commonplace. How might designers support such human values? We interviewed 27 public sector machine learning practitioners across 5 OECD countries regarding challenges understanding and imbuing public values into their work. The results suggest a disconnect between organisational and institutional realities, constraints and needs, and those addressed by current research into usable, transparent and 'discrimination-aware' machine learning---absences likely to undermine practical initiatives unless addressed. We see design opportunities in this disconnect, such as in supporting the tracking of concept drift in secondary data sources, and in building usable transparency tools to identify risks and incorporate domain knowledge, aimed both at managers and at the 'street-level bureaucrats' on the frontlines of public service. We conclude by outlining ethical challenges and future directions for collaboration in these high-stakes applications.