Balancing Logit Variation for Long-tailed Semantic Segmentation

TL;DR

This paper proposes a simple yet effective method to balance feature distributions across categories in long-tailed semantic segmentation by introducing category-dependent variations during training, improving overall performance.

Contribution

It introduces a novel category-wise variation technique to balance feature space in long-tailed segmentation, compatible with existing methods and datasets.

Findings

Improves segmentation accuracy on long-tailed datasets

Enhances generalizability across different models and datasets

Plug-in method boosts performance of state-of-the-art approaches

Abstract

Semantic segmentation usually suffers from a long-tail data distribution. Due to the imbalanced number of samples across categories, the features of those tail classes may get squeezed into a narrow area in the feature space. Towards a balanced feature distribution, we introduce category-wise variation into the network predictions in the training phase such that an instance is no longer projected to a feature point, but a small region instead. Such a perturbation is highly dependent on the category scale, which appears as assigning smaller variation to head classes and larger variation to tail classes. In this way, we manage to close the gap between the feature areas of different categories, resulting in a more balanced representation. It is noteworthy that the introduced variation is discarded at the inference stage to facilitate a confident prediction. Although with an embarrassingly simple implementation, our method manifests itself in strong generalizability to various datasets and task settings. Extensive experiments suggest that our plug-in design lends itself well to a range of state-of-the-art approaches and boosts the performance on top of them.