Ethics Sheets for AI Tasks

TL;DR

This paper advocates for the development of Ethics Sheets for AI Tasks to systematically document ethical considerations at the task level, aiming to improve awareness and accountability in AI development.

Contribution

It introduces a new framework for ethical evaluation called Ethics Sheets for AI Tasks, including a template with 50 considerations, exemplified through emotion recognition.

Findings

Ethics sheets can guide ethical decision-making in AI development.

A template with 50 considerations helps standardize ethical assessments.

Ethics sheets facilitate proactive engagement with ethical issues.

Abstract

Several high-profile events, such as the mass testing of emotion recognition systems on vulnerable sub-populations and using question answering systems to make moral judgments, have highlighted how technology will often lead to more adverse outcomes for those that are already marginalized. At issue here are not just individual systems and datasets, but also the AI tasks themselves. In this position paper, I make a case for thinking about ethical considerations not just at the level of individual models and datasets, but also at the level of AI tasks. I will present a new form of such an effort, Ethics Sheets for AI Tasks, dedicated to fleshing out the assumptions and ethical considerations hidden in how a task is commonly framed and in the choices we make regarding the data, method, and evaluation. I will also present a template for ethics sheets with 50 ethical considerations, using the task of emotion recognition as a running example. Ethics sheets are a mechanism to engage with and document ethical considerations before building datasets and systems. Similar to survey articles, a small number of ethics sheets can serve numerous researchers and developers.