Correlation Loss: Enforcing Correlation between Classification and Localization

TL;DR

This paper introduces Correlation Loss, a new loss function that directly optimizes the correlation between classification and localization tasks in object detectors, leading to significant performance improvements across various models.

Contribution

It provides an analysis of how correlation affects detector performance and proposes a novel Correlation Loss to enhance detection accuracy, including for NMS-free models.

Findings

Correlation Loss improves Sparse R-CNN by 1.6 AP on COCO

Correlation Loss yields 1.8 AP gain on Cityscapes

State-of-the-art 51.0 AP achieved on COCO test-dev

Abstract

Object detectors are conventionally trained by a weighted sum of classification and localization losses. Recent studies (e.g., predicting IoU with an auxiliary head, Generalized Focal Loss, Rank & Sort Loss) have shown that forcing these two loss terms to interact with each other in non-conventional ways creates a useful inductive bias and improves performance. Inspired by these works, we focus on the correlation between classification and localization and make two main contributions: (i) We provide an analysis about the effects of correlation between classification and localization tasks in object detectors. We identify why correlation affects the performance of various NMS-based and NMS-free detectors, and we devise measures to evaluate the effect of correlation and use them to analyze common detectors. (ii) Motivated by our observations, e.g., that NMS-free detectors can also benefit from correlation, we propose Correlation Loss, a novel plug-in loss function that improves the performance of various object detectors by directly optimizing correlation coefficients: E.g., Correlation Loss on Sparse R-CNN, an NMS-free method, yields 1.6 AP gain on COCO and 1.8 AP gain on Cityscapes dataset. Our best model on Sparse R-CNN reaches 51.0 AP without test-time augmentation on COCO test-dev, reaching state-of-the-art. Code is available at https://github.com/fehmikahraman/CorrLoss