Pair DETR: Contrastive Learning Speeds Up DETR Training

TL;DR

Pair DETR introduces a contrastive learning approach that detects objects as paired keypoints, significantly accelerating training convergence and improving detection accuracy on the MS COCO dataset.

Contribution

The paper proposes a novel method using paired keypoints and contrastive learning to speed up DETR training without changing its architecture.

Findings

Converges at least 10x faster than original DETR.

Converges 1.5x faster than Conditional DETR.

Achieves higher Average Precision scores.

Abstract

The DETR object detection approach applies the transformer encoder and decoder architecture to detect objects and achieves promising performance. In this paper, we present a simple approach to address the main problem of DETR, the slow convergence, by using representation learning technique. In this approach, we detect an object bounding box as a pair of keypoints, the top-left corner and the center, using two decoders. By detecting objects as paired keypoints, the model builds up a joint classification and pair association on the output queries from two decoders. For the pair association we propose utilizing contrastive self-supervised learning algorithm without requiring specialized architecture. Experimental results on MS COCO dataset show that Pair DETR can converge at least 10x faster than original DETR and 1.5x faster than Conditional DETR during training, while having consistently higher Average Precision scores.