Inverse Image Frequency for Long-tailed Image Recognition

TL;DR

The paper introduces Inverse Image Frequency (IIF), a de-biasing technique for long-tailed image recognition that improves accuracy and reduces false positives across various benchmarks by adjusting logits based on class frequency.

Contribution

It proposes a novel multiplicative margin adjustment method called IIF that effectively mitigates bias in long-tailed image classification and segmentation tasks.

Findings

IIF outperforms existing methods on multiple long-tailed benchmarks.

IIF achieves 55.8% top-1 accuracy on ImageNet-LT with ResNet50.

IIF reduces false positives in long-tailed instance segmentation.

Abstract

The long-tailed distribution is a common phenomenon in the real world. Extracted large scale image datasets inevitably demonstrate the long-tailed property and models trained with imbalanced data can obtain high performance for the over-represented categories, but struggle for the under-represented categories, leading to biased predictions and performance degradation. To address this challenge, we propose a novel de-biasing method named Inverse Image Frequency (IIF). IIF is a multiplicative margin adjustment transformation of the logits in the classification layer of a convolutional neural network. Our method achieves stronger performance than similar works and it is especially useful for downstream tasks such as long-tailed instance segmentation as it produces fewer false positive detections. Our extensive experiments show that IIF surpasses the state of the art on many long-tailed benchmarks such as ImageNet-LT, CIFAR-LT, Places-LT and LVIS, reaching 55.8% top-1 accuracy with ResNet50 on ImageNet-LT and 26.2% segmentation AP with MaskRCNN on LVIS. Code available at https://github.com/kostas1515/iif