Bayesian Detector Combination for Object Detection with Crowdsourced Annotations

TL;DR

This paper introduces a Bayesian Detector Combination framework that effectively trains object detectors using noisy crowdsourced annotations, automatically inferring annotator quality and outperforming existing methods.

Contribution

The paper proposes a novel, model-agnostic Bayesian framework for object detection with crowdsourced data, capable of inferring annotator quality without prior knowledge.

Findings

BDC outperforms state-of-the-art methods on real datasets.

Synthetic datasets enable scalable, consistent evaluation.

BDC seamlessly integrates with existing object detection models.

Abstract

Acquiring fine-grained object detection annotations in unconstrained images is time-consuming, expensive, and prone to noise, especially in crowdsourcing scenarios. Most prior object detection methods assume accurate annotations; A few recent works have studied object detection with noisy crowdsourced annotations, with evaluation on distinct synthetic crowdsourced datasets of varying setups under artificial assumptions. To address these algorithmic limitations and evaluation inconsistency, we first propose a novel Bayesian Detector Combination (BDC) framework to more effectively train object detectors with noisy crowdsourced annotations, with the unique ability of automatically inferring the annotators' label qualities. Unlike previous approaches, BDC is model-agnostic, requires no prior knowledge of the annotators' skill level, and seamlessly integrates with existing object detection models. Due to the scarcity of real-world crowdsourced datasets, we introduce large synthetic datasets by simulating varying crowdsourcing scenarios. This allows consistent evaluation of different models at scale. Extensive experiments on both real and synthetic crowdsourced datasets show that BDC outperforms existing state-of-the-art methods, demonstrating its superiority in leveraging crowdsourced data for object detection. Our code and data are available at https://github.com/zhiqin1998/bdc.