Relieving Long-tailed Instance Segmentation via Pairwise Class Balance

TL;DR

This paper introduces a novel Pairwise Class Balance (PCB) method that uses a confusion matrix to reduce class bias in long-tailed instance segmentation, achieving state-of-the-art results.

Contribution

The paper proposes a confusion matrix-based PCB approach with iterative learning for effective bias reduction in long-tailed segmentation.

Findings

PCB achieves state-of-the-art performance on LVIS dataset.

The method generalizes well across different architectures.

PCB effectively reduces class bias and improves segmentation accuracy.

Abstract

Long-tailed instance segmentation is a challenging task due to the extreme imbalance of training samples among classes. It causes severe biases of the head classes (with majority samples) against the tailed ones. This renders "how to appropriately define and alleviate the bias" one of the most important issues. Prior works mainly use label distribution or mean score information to indicate a coarse-grained bias. In this paper, we explore to excavate the confusion matrix, which carries the fine-grained misclassification details, to relieve the pairwise biases, generalizing the coarse one. To this end, we propose a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences. PCB generates fightback soft labels for regularization during training. Besides, an iterative learning paradigm is developed to support a progressive and smooth regularization in such debiasing. PCB can be plugged and played to any existing method as a complement. Experimental results on LVIS demonstrate that our method achieves state-of-the-art performance without bells and whistles. Superior results across various architectures show the generalization ability. The code and trained models are available at https://github.com/megvii-research/PCB.