Points as Queries: Weakly Semi-supervised Object Detection by Points

TL;DR

This paper introduces a new weakly semi-supervised object detection setting using point annotations, and proposes Point DETR, a detector that effectively exploits point annotations to improve detection performance with less fully labeled data.

Contribution

The paper presents a novel point-annotated semi-supervised detection setting and a new detector, Point DETR, that extends DETR with a point encoder to better utilize point annotations.

Findings

Point DETR outperforms baseline methods on MS-COCO.

Using 20% fully labeled data, Point DETR achieves 33.3 AP, surpassing FCOS by 2.0 AP.

Point annotations improve various AR metrics by over 10 points.

Abstract

We propose a novel point annotated setting for the weakly semi-supervised object detection task, in which the dataset comprises small fully annotated images and large weakly annotated images by points. It achieves a balance between tremendous annotation burden and detection performance. Based on this setting, we analyze existing detectors and find that these detectors have difficulty in fully exploiting the power of the annotated points. To solve this, we introduce a new detector, Point DETR, which extends DETR by adding a point encoder. Extensive experiments conducted on MS-COCO dataset in various data settings show the effectiveness of our method. In particular, when using 20% fully labeled data from COCO, our detector achieves a promising performance, 33.3 AP, which outperforms a strong baseline (FCOS) by 2.0 AP, and we demonstrate the point annotations bring over 10 points in various AR metrics.