VarifocalNet: An IoU-aware Dense Object Detector

TL;DR

VarifocalNet introduces an IoU-aware classification score and a new loss function to improve dense object detection accuracy, achieving state-of-the-art results on MS COCO by better ranking candidate detections.

Contribution

The paper proposes a novel IoU-aware classification score, Varifocal Loss, and a star-shaped bounding box feature for improved dense object detection.

Findings

VFNet surpasses baseline by ~2.0 AP on MS COCO

Achieves 55.1 AP on COCO test-dev with Res2Net-101-DCN

State-of-the-art performance among object detectors

Abstract

Accurately ranking the vast number of candidate detections is crucial for dense object detectors to achieve high performance. Prior work uses the classification score or a combination of classification and predicted localization scores to rank candidates. However, neither option results in a reliable ranking, thus degrading detection performance. In this paper, we propose to learn an Iou-aware Classification Score (IACS) as a joint representation of object presence confidence and localization accuracy. We show that dense object detectors can achieve a more accurate ranking of candidate detections based on the IACS. We design a new loss function, named Varifocal Loss, to train a dense object detector to predict the IACS, and propose a new star-shaped bounding box feature representation for IACS prediction and bounding box refinement. Combining these two new components and a bounding box refinement branch, we build an IoU-aware dense object detector based on the FCOS+ATSS architecture, that we call VarifocalNet or VFNet for short. Extensive experiments on MS COCO show that our VFNet consistently surpasses the strong baseline by $\sim$2.0 AP with different backbones. Our best model VFNet-X-1200 with Res2Net-101-DCN achieves a single-model single-scale AP of 55.1 on COCO test-dev, which is state-of-the-art among various object detectors.Code is available at https://github.com/hyz-xmaster/VarifocalNet .