RepPoints: Point Set Representation for Object Detection

TL;DR

RepPoints introduces a set of learned sample points for object detection, providing finer localization than bounding boxes and eliminating the need for anchors, achieving competitive performance on COCO benchmark.

Contribution

The paper proposes RepPoints, a novel anchor-free object representation using learnable points that improve localization and recognition accuracy.

Findings

Achieves 46.5 AP on COCO test-dev with ResNet-101.

Eliminates the need for anchor boxes in object detection.

Performs comparably to state-of-the-art anchor-based methods.

Abstract

Modern object detectors rely heavily on rectangular bounding boxes, such as anchors, proposals and the final predictions, to represent objects at various recognition stages. The bounding box is convenient to use but provides only a coarse localization of objects and leads to a correspondingly coarse extraction of object features. In this paper, we present \textbf{RepPoints} (representative points), a new finer representation of objects as a set of sample points useful for both localization and recognition. Given ground truth localization and recognition targets for training, RepPoints learn to automatically arrange themselves in a manner that bounds the spatial extent of an object and indicates semantically significant local areas. They furthermore do not require the use of anchors to sample a space of bounding boxes. We show that an anchor-free object detector based on RepPoints can be as effective as the state-of-the-art anchor-based detection methods, with 46.5 AP and 67.4 $AP_{50}$ on the COCO test-dev detection benchmark, using ResNet-101 model. Code is available at https://github.com/microsoft/RepPoints.