A Study on Self-Supervised Object Detection Pretraining

TL;DR

This paper investigates self-supervised pretraining methods for object detection, proposing a general framework and auxiliary tasks, but finds limited improvements over existing approaches in downstream detection performance.

Contribution

It introduces a unified framework for dense representation learning in self-supervised object detection and evaluates auxiliary tasks, highlighting their limited impact.

Findings

Multiple views are less effective than in instance-level learning.

Auxiliary tasks did not improve detection performance.

The framework is robust to hyperparameter choices.

Abstract

In this work, we study different approaches to self-supervised pretraining of object detection models. We first design a general framework to learn a spatially consistent dense representation from an image, by randomly sampling and projecting boxes to each augmented view and maximizing the similarity between corresponding box features. We study existing design choices in the literature, such as box generation, feature extraction strategies, and using multiple views inspired by its success on instance-level image representation learning techniques. Our results suggest that the method is robust to different choices of hyperparameters, and using multiple views is not as effective as shown for instance-level image representation learning. We also design two auxiliary tasks to predict boxes in one view from their features in the other view, by (1) predicting boxes from the sampled set by using a contrastive loss, and (2) predicting box coordinates using a transformer, which potentially benefits downstream object detection tasks. We found that these tasks do not lead to better object detection performance when finetuning the pretrained model on labeled data.