ComplETR: Reducing the cost of annotations for object detection in dense scenes with vision transformers

TL;DR

ComplETR is a novel framework that leverages vision transformers to complete missing annotations in dense scene datasets, significantly reducing annotation costs while improving detection performance across various models.

Contribution

It introduces a DETR-based method that explicitly completes missing annotations using patch information, outperforming state-of-the-art methods and enhancing multiple object detectors.

Findings

Outperforms Soft Sampling and Unbiased Teacher methods

Improves detection accuracy across multiple models

Reduces annotation effort in dense scenes

Abstract

Annotating bounding boxes for object detection is expensive, time-consuming, and error-prone. In this work, we propose a DETR based framework called ComplETR that is designed to explicitly complete missing annotations in partially annotated dense scene datasets. This reduces the need to annotate every object instance in the scene thereby reducing annotation cost. ComplETR augments object queries in DETR decoder with patch information of objects in the image. Combined with a matching loss, it can effectively find objects that are similar to the input patch and complete the missing annotations. We show that our framework outperforms the state-of-the-art methods such as Soft Sampling and Unbiased Teacher by itself, while at the same time can be used in conjunction with these methods to further improve their performance. Our framework is also agnostic to the choice of the downstream object detectors; we show performance improvement for several popular detectors such as Faster R-CNN, Cascade R-CNN, CenterNet2, and Deformable DETR on multiple dense scene datasets.