Scalable Class-Centric Visual Interactive Labeling

TL;DR

cVIL introduces a class-centric visual interactive labeling approach that significantly improves scalability and efficiency in labeling large, complex datasets by reducing cognitive load and effort for annotators.

Contribution

The paper presents a novel class-centric labeling methodology and a visual analytics interface that outperform traditional instance-centric methods in large-scale data annotation tasks.

Findings

cVIL improves labeling efficiency in user studies

The interface reduces cognitive load for annotators

cVIL scales better with large, complex datasets

Abstract

Large unlabeled datasets demand efficient and scalable data labeling solutions, in particular when the number of instances and classes is large. This leads to significant visual scalability challenges and imposes a high cognitive load on the users. Traditional instance-centric labeling methods, where (single) instances are labeled in each iteration struggle to scale effectively in these scenarios. To address these challenges, we introduce cVIL, a Class-Centric Visual Interactive Labeling methodology designed for interactive visual data labeling. By shifting the paradigm from assigning-classes-to-instances to assigning-instances-to-classes, cVIL reduces labeling effort and enhances efficiency for annotators working with large, complex and class-rich datasets. We propose a novel visual analytics labeling interface built on top of the conceptual cVIL workflow, enabling improved scalability over traditional visual labeling. In a user study, we demonstrate that cVIL can improve labeling efficiency and user satisfaction over instance-centric interfaces. The effectiveness of cVIL is further demonstrated through a usage scenario, showcasing its potential to alleviate cognitive load and support experts in managing extensive labeling tasks efficiently.