FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation

TL;DR

FASA is a versatile method that enhances long-tailed instance segmentation by adaptively augmenting features and sampling, leading to improved performance especially on rare classes without complex loss modifications.

Contribution

Introducing FASA, a simple, adaptive feature augmentation and sampling method that improves long-tailed segmentation without elaborate loss design or inter-class transfer learning.

Findings

FASA improves segmentation accuracy on rare classes.

FASA achieves state-of-the-art results in long-tailed classification.

FASA is fast, generic, and easily integrated into existing frameworks.

Abstract

Recent methods for long-tailed instance segmentation still struggle on rare object classes with few training data. We propose a simple yet effective method, Feature Augmentation and Sampling Adaptation (FASA), that addresses the data scarcity issue by augmenting the feature space especially for rare classes. Both the Feature Augmentation (FA) and feature sampling components are adaptive to the actual training status -- FA is informed by the feature mean and variance of observed real samples from past iterations, and we sample the generated virtual features in a loss-adapted manner to avoid over-fitting. FASA does not require any elaborate loss design, and removes the need for inter-class transfer learning that often involves large cost and manually-defined head/tail class groups. We show FASA is a fast, generic method that can be easily plugged into standard or long-tailed segmentation frameworks, with consistent performance gains and little added cost. FASA is also applicable to other tasks like long-tailed classification with state-of-the-art performance.