Reconcile Prediction Consistency for Balanced Object Detection

TL;DR

This paper introduces a Harmonic loss function to synchronize the training of classification and localization branches in object detectors, improving prediction consistency and overall detection accuracy.

Contribution

The proposed Harmonic loss and Harmonic IoU loss enable joint optimization of classification and localization, enhancing detection performance especially for irregular and occluded objects.

Findings

Improved detection accuracy on PASCAL VOC and MS COCO benchmarks.

Enhanced prediction consistency between classification and localization.

Achieved state-of-the-art results with the proposed method.

Abstract

Classification and regression are two pillars of object detectors. In most CNN-based detectors, these two pillars are optimized independently. Without direct interactions between them, the classification loss and the regression loss can not be optimized synchronously toward the optimal direction in the training phase. This clearly leads to lots of inconsistent predictions with high classification score but low localization accuracy or low classification score but high localization accuracy in the inference phase, especially for the objects of irregular shape and occlusion, which severely hurts the detection performance of existing detectors after NMS. To reconcile prediction consistency for balanced object detection, we propose a Harmonic loss to harmonize the optimization of classification branch and localization branch. The Harmonic loss enables these two branches to supervise and promote each other during training, thereby producing consistent predictions with high co-occurrence of top classification and localization in the inference phase. Furthermore, in order to prevent the localization loss from being dominated by outliers during training phase, a Harmonic IoU loss is proposed to harmonize the weight of the localization loss of different IoU-level samples. Comprehensive experiments on benchmarks PASCAL VOC and MS COCO demonstrate the generality and effectiveness of our model for facilitating existing object detectors to state-of-the-art accuracy.