Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors
Hong Shen, Alicia DeVos, Motahhare Eslami, Kenneth Holstein

TL;DR
This paper highlights the importance of everyday users in detecting harmful algorithmic behaviors through their routine interactions, emphasizing the need to integrate bottom-up auditing with formal methods for more effective oversight.
Contribution
It introduces the concept of everyday algorithm auditing, analyzing real-world cases and proposing design considerations to enhance user-driven detection of problematic behaviors.
Findings
Users can detect issues that formal audits may miss
Everyday interactions reveal diverse harmful behaviors
Designing platforms for user auditing can improve system oversight
Abstract
A growing body of literature has proposed formal approaches to audit algorithmic systems for biased and harmful behaviors. While formal auditing approaches have been greatly impactful, they often suffer major blindspots, with critical issues surfacing only in the context of everyday use once systems are deployed. Recent years have seen many cases in which everyday users of algorithmic systems detect and raise awareness about harmful behaviors that they encounter in the course of their everyday interactions with these systems. However, to date little academic attention has been granted to these bottom-up, user-driven auditing processes. In this paper, we propose and explore the concept of everyday algorithm auditing, a process in which users detect, understand, and interrogate problematic machine behaviors via their day-to-day interactions with algorithmic systems. We argue that everyday…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
