Spatial Consistency Loss for Training Multi-Label Classifiers from Single-Label Annotations

TL;DR

This paper introduces a spatial consistency loss to effectively train multi-label classifiers from single-label annotations, improving performance by maintaining prediction stability over training epochs and across spatial feature maps.

Contribution

The paper proposes a novel spatial consistency loss that enhances weakly supervised multi-label classification by enforcing temporal and spatial prediction consistency during training.

Findings

Spatial consistency loss improves multi-label mAP on MS-COCO and Pascal VOC.

The method overcomes data augmentation limitations by recovering supervision signals.

Enhanced multi-label classification performance on ImageNet-1K using the proposed approach.

Abstract

As natural images usually contain multiple objects, multi-label image classification is more applicable "in the wild" than single-label classification. However, exhaustively annotating images with every object of interest is costly and time-consuming. We aim to train multi-label classifiers from single-label annotations only. We show that adding a consistency loss, ensuring that the predictions of the network are consistent over consecutive training epochs, is a simple yet effective method to train multi-label classifiers in a weakly supervised setting. We further extend this approach spatially, by ensuring consistency of the spatial feature maps produced over consecutive training epochs, maintaining per-class running-average heatmaps for each training image. We show that this spatial consistency loss further improves the multi-label mAP of the classifiers. In addition, we show that this method overcomes shortcomings of the "crop" data-augmentation by recovering correct supervision signal even when most of the single ground truth object is cropped out of the input image by the data augmentation. We demonstrate gains of the consistency and spatial consistency losses over the binary cross-entropy baseline, and over competing methods, on MS-COCO and Pascal VOC. We also demonstrate improved multi-label classification mAP on ImageNet-1K using the ReaL multi-label validation set.