Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice

TL;DR

This paper explores industry practitioners' current practices, challenges, and opportunities in user-engaged algorithm auditing, highlighting organizational obstacles and potential research directions to improve effectiveness and mitigate risks.

Contribution

It provides empirical insights into real-world challenges and organizational factors affecting user-engaged algorithm auditing in industry settings.

Findings

Practitioners face challenges in recruiting and incentivizing user auditors.

Organizational obstacles hinder effective user-engaged auditing.

There are opportunities for HCI research to enhance audit practices and address risks.

Abstract

Recent years have seen growing interest among both researchers and practitioners in user-engaged approaches to algorithm auditing, which directly engage users in detecting problematic behaviors in algorithmic systems. However, we know little about industry practitioners' current practices and challenges around user-engaged auditing, nor what opportunities exist for them to better leverage such approaches in practice. To investigate, we conducted a series of interviews and iterative co-design activities with practitioners who employ user-engaged auditing approaches in their work. Our findings reveal several challenges practitioners face in appropriately recruiting and incentivizing user auditors, scaffolding user audits, and deriving actionable insights from user-engaged audit reports. Furthermore, practitioners shared organizational obstacles to user-engaged auditing, surfacing a complex relationship between practitioners and user auditors. Based on these findings, we discuss opportunities for future HCI research to help realize the potential (and the mitigate risks) of user-engaged auditing in industry practice.