Orthogonal Uncertainty Representation of Data Manifold for Robust Long-Tailed Learning

TL;DR

This paper introduces Orthogonal Uncertainty Representation (OUR), a novel method to enhance model robustness in long-tailed learning scenarios by addressing bias on noisy data manifolds, without extra data generation.

Contribution

The paper proposes a new orthogonal uncertainty representation technique and an end-to-end training strategy that improves robustness in long-tailed distributions, compatible with existing methods.

Findings

Significant robustness improvements on long-tailed datasets.

Consistent performance gains when combined with other methods.

Efficient training without additional data generation.

Abstract

In scenarios with long-tailed distributions, the model's ability to identify tail classes is limited due to the under-representation of tail samples. Class rebalancing, information augmentation, and other techniques have been proposed to facilitate models to learn the potential distribution of tail classes. The disadvantage is that these methods generally pursue models with balanced class accuracy on the data manifold, while ignoring the ability of the model to resist interference. By constructing noisy data manifold, we found that the robustness of models trained on unbalanced data has a long-tail phenomenon. That is, even if the class accuracy is balanced on the data domain, it still has bias on the noisy data manifold. However, existing methods cannot effectively mitigate the above phenomenon, which makes the model vulnerable in long-tailed scenarios. In this work, we propose an Orthogonal Uncertainty Representation (OUR) of feature embedding and an end-to-end training strategy to improve the long-tail phenomenon of model robustness. As a general enhancement tool, OUR has excellent compatibility with other methods and does not require additional data generation, ensuring fast and efficient training. Comprehensive evaluations on long-tailed datasets show that our method significantly improves the long-tail phenomenon of robustness, bringing consistent performance gains to other long-tailed learning methods.