Crowdsourcing Impacts: Exploring the Utility of Crowds for Anticipating Societal Impacts of Algorithmic Decision Making

TL;DR

This paper demonstrates that crowdsourcing can effectively anticipate societal impacts of algorithms by leveraging diverse perspectives to identify impact areas, ethical concerns, and societal domains, thereby complementing existing assessment methods.

Contribution

It introduces crowdsourcing as a novel participatory foresight approach for predicting societal impacts of algorithms, highlighting its effectiveness and potential for uncovering diverse impact issues.

Findings

Crowdsourcing uncovers a wide range of societal impact issues.

It reveals patterns and connections between different impact areas.

The method effectively leverages cognitive diversity to anticipate impacts.

Abstract

With the increasing pervasiveness of algorithms across industry and government, a growing body of work has grappled with how to understand their societal impact and ethical implications. Various methods have been used at different stages of algorithm development to encourage researchers and designers to consider the potential societal impact of their research. An understudied yet promising area in this realm is using participatory foresight to anticipate these different societal impacts. We employ crowdsourcing as a means of participatory foresight to uncover four different types of impact areas based on a set of governmental algorithmic decision making tools: (1) perceived valence, (2) societal domains, (3) specific abstract impact types, and (4) ethical algorithm concerns. Our findings suggest that this method is effective at leveraging the cognitive diversity of the crowd to uncover a range of issues. We further analyze the complexities within the interaction of the impact areas identified to demonstrate how crowdsourcing can illuminate patterns around the connections between impacts. Ultimately this work establishes crowdsourcing as an effective means of anticipating algorithmic impact which complements other approaches towards assessing algorithms in society by leveraging participatory foresight and cognitive diversity.