Whose AI Dream? In search of the aspiration in data annotation

TL;DR

This paper explores data annotation practices in India from annotators' perspectives, revealing power dynamics and proposing ways to improve working conditions and participation in AI development.

Contribution

It offers a grounded, qualitative analysis of data annotation work, emphasizing organizational influence and annotator aspirations, which has been underexplored in prior research.

Findings

Annotators' work is shaped by organizational interests and power structures.

Data annotation is more than technical work; it involves systemic power relations.

Implications for improving annotation practices and supporting annotator well-being.

Abstract

This paper present the practice of data annotation from the perspective of the annotators. Data is fundamental to ML models. This paper investigates the work practices concerning data annotation as performed in the industry, in India. Previous investigations have largely focused on annotator subjectivity, bias and efficiency. We present a wider perspective of the data annotation, following a grounded approach, we conducted three sets of interviews with 25 annotators, 10 industry experts and 12 ML practitioners. Our results show that the work of annotators is dictated by the interests, priorities and values of others above their station. More than technical, we contend that data annotation is a systematic exercise of power through organizational structure and practice. We propose a set of implications for how we can cultivate and encourage better practice to balance the tension between the need for high quality data at low cost and the annotator aspiration for well being, career perspective, and active participation in building the AI dream.