Single-Pass Object-Focused Data Selection

TL;DR

This paper introduces Object-Focused Data Selection (OFDS), a method leveraging object-level features from foundation models to improve data selection for labeling, significantly enhancing object detection and segmentation performance.

Contribution

OFDS is a novel approach that uses object-level features for single-pass data selection, outperforming prior image-level methods across various tasks and domains.

Findings

OFDS consistently outperforms random selection and baselines.

Combining human labels from OFDS with autolabels improves results.

Using OFDS for initial active learning selection yields consistent gains.

Abstract

While unlabeled image data is often plentiful, the costs of high-quality labels pose an important practical challenge: Which images should one select for labeling to use the annotation budget for a particular target task most effectively? To address this problem, we focus on single-pass data selection, which refers to the process of selecting all data to be annotated at once before training a downstream model. Prior methods for single-pass data selection rely on image-level representations and fail to reliably outperform random selection for object detection and segmentation. We propose Object-Focused Data Selection (OFDS) which leverages object-level features from foundation models and ensures semantic coverage of all target classes. In extensive experiments across tasks and target domains, OFDS consistently outperforms random selection and all baselines. The best results for constrained annotation budgets are obtained by combining human labels from OFDS with autolabels from foundation models. Moreover, using OFDS to select the initial labeled set for active learning yields consistent improvements