Conscientious Classification: A Data Scientist's Guide to Discrimination-Aware Classification

TL;DR

This paper guides data scientists on how to recognize, measure, and mitigate discrimination in machine learning systems, emphasizing proactive strategies to prevent perpetuating societal biases.

Contribution

It introduces a discrimination-aware framework within the data science process, providing taxonomy, measurement methods, and mitigation strategies for reducing bias.

Findings

Provides a taxonomy of practices leading to discrimination

Surveys methods for measuring discrimination

Suggests process augmentations to mitigate bias

Abstract

Recent research has helped to cultivate growing awareness that machine learning systems fueled by big data can create or exacerbate troubling disparities in society. Much of this research comes from outside of the practicing data science community, leaving its members with little concrete guidance to proactively address these concerns. This article introduces issues of discrimination to the data science community on its own terms. In it, we tour the familiar data mining process while providing a taxonomy of common practices that have the potential to produce unintended discrimination. We also survey how discrimination is commonly measured, and suggest how familiar development processes can be augmented to mitigate systems' discriminatory potential. We advocate that data scientists should be intentional about modeling and reducing discriminatory outcomes. Without doing so, their efforts will result in perpetuating any systemic discrimination that may exist, but under a misleading veil of data-driven objectivity.