Exploring Data Pipelines through the Process Lens: a Reference Model forComputer Vision

TL;DR

This paper introduces a process-oriented reference model for computer vision data pipelines, aiming to enhance understanding of dataset creation, evolution, and use to identify harms and improve transparency.

Contribution

It proposes a novel process lens and a reference model for CV data pipelines, moving beyond object-focused analyses to systematize harm exploration.

Findings

Preliminary reference model of CV data pipelines

Process lens can reveal understudied issues in datasets

Potential to improve transparency and harm detection

Abstract

Researchers have identified datasets used for training computer vision (CV) models as an important source of hazardous outcomes, and continue to examine popular CV datasets to expose their harms. These works tend to treat datasets as objects, or focus on particular steps in data production pipelines. We argue here that we could further systematize our analysis of harms by examining CV data pipelines through a process-oriented lens that captures the creation, the evolution and use of these datasets. As a step towards cultivating a process-oriented lens, we embarked on an empirical study of CV data pipelines informed by the field of method engineering. We present here a preliminary result: a reference model of CV data pipelines. Besides exploring the questions that this endeavor raises, we discuss how the process lens could support researchers in discovering understudied issues, and could help practitioners in making their processes more transparent.