Unpacking the Expressed Consequences of AI Research in Broader Impact Statements

TL;DR

This paper analyzes how AI researchers articulate potential impacts of their work in NeurIPS 2020 statements, revealing themes, concerns, and suggestions for improving impact disclosures.

Contribution

It provides a qualitative analysis of impact statements, identifying key themes and proposing ways to enhance their effectiveness in future conferences.

Findings

Impact statements often mention bias, environment, privacy, and labor.

Researchers express uncertainty and call for mitigation strategies.

Thematic categories include valence, specificity, and impact areas.

Abstract

The computer science research community and the broader public have become increasingly aware of negative consequences of algorithmic systems. In response, the top-tier Neural Information Processing Systems (NeurIPS) conference for machine learning and artificial intelligence research required that authors include a statement of broader impact to reflect on potential positive and negative consequences of their work. We present the results of a qualitative thematic analysis of a sample of statements written for the 2020 conference. The themes we identify broadly fall into categories related to how consequences are expressed (e.g., valence, specificity, uncertainty), areas of impacts expressed (e.g., bias, the environment, labor, privacy), and researchers' recommendations for mitigating negative consequences in the future. In light of our results, we offer perspectives on how the broader impact statement can be implemented in future iterations to better align with potential goals.