Localize to Classify and Classify to Localize: Mutual Guidance in Object Detection

TL;DR

This paper introduces a mutual guidance strategy for object detection that dynamically improves anchor matching by jointly optimizing localization and classification tasks, leading to better performance on standard datasets.

Contribution

It proposes a novel anchor matching criterion that uses mutual guidance between localization and classification during training, enhancing detection accuracy.

Findings

Improved detection performance on PASCAL VOC and MS COCO datasets.

Demonstrated effectiveness across various deep learning architectures.

Showed generality and simplicity of the proposed method.

Abstract

Most deep learning object detectors are based on the anchor mechanism and resort to the Intersection over Union (IoU) between predefined anchor boxes and ground truth boxes to evaluate the matching quality between anchors and objects. In this paper, we question this use of IoU and propose a new anchor matching criterion guided, during the training phase, by the optimization of both the localization and the classification tasks: the predictions related to one task are used to dynamically assign sample anchors and improve the model on the other task, and vice versa. Despite the simplicity of the proposed method, our experiments with different state-of-the-art deep learning architectures on PASCAL VOC and MS COCO datasets demonstrate the effectiveness and generality of our Mutual Guidance strategy.