The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment

TL;DR

This paper analyzes the process and factors leading organizations to abandon harmful algorithms, identifying a six-phase process and key social-technical influences to inform future mitigation strategies.

Contribution

It introduces the concept of algorithm abandonment, characterizes its dynamics through real-world case analysis, and proposes a six-phase framework called the '6 D's of abandonment.'

Findings

Identification of six iterative phases in abandonment process

Key social and technical factors influencing abandonment decisions

Framework to guide stakeholders in mitigating algorithmic harms

Abstract

As more algorithmic systems have come under scrutiny for their potential to inflict societal harms, an increasing number of organizations that hold power over harmful algorithms have chosen (or were required under the law) to abandon them. While social movements and calls to abandon harmful algorithms have emerged across application domains, little academic attention has been paid to studying abandonment as a means to mitigate algorithmic harms. In this paper, we take a first step towards conceptualizing "algorithm abandonment" as an organization's decision to stop designing, developing, or using an algorithmic system due to its (potential) harms. We conduct a thematic analysis of real-world cases of algorithm abandonment to characterize the dynamics leading to this outcome. Our analysis of 40 cases reveals that campaigns to abandon an algorithm follow a common process of six iterative phases: discovery, diagnosis, dissemination, dialogue, decision, and death, which we term the "6 D's of abandonment". In addition, we highlight key factors that facilitate (or prohibit) abandonment, which include characteristics of both the technical and social systems that the algorithm is embedded within. We discuss implications for several stakeholders, including proprietors and technologists who have the power to influence an algorithm's (dis)continued use, FAccT researchers, and policymakers.