ViRN: Variational Inference and Distribution Trilateration for Long-Tailed Continual Representation Learning

TL;DR

ViRN introduces a novel continual learning framework combining variational inference and distribution trilateration to effectively handle long-tailed data distributions, improving accuracy on diverse benchmarks.

Contribution

It presents a new approach integrating VI with distributional trilateration to enhance long-tailed continual learning, addressing class imbalance and sample scarcity.

Findings

Achieves 10.24% higher average accuracy than state-of-the-art methods.

Effectively models class distributions to mitigate bias toward frequent classes.

Improves tail-class representation alignment through Wasserstein-based geometric methods.

Abstract

Continual learning (CL) with long-tailed data distributions remains a critical challenge for real-world AI systems, where models must sequentially adapt to new classes while retaining knowledge of old ones, despite severe class imbalance. Existing methods struggle to balance stability and plasticity, often collapsing under extreme sample scarcity. To address this, we propose ViRN, a novel CL framework that integrates variational inference (VI) with distributional trilateration for robust long-tailed learning. First, we model class-conditional distributions via a Variational Autoencoder to mitigate bias toward head classes. Second, we reconstruct tail-class distributions via Wasserstein distance-based neighborhood retrieval and geometric fusion, enabling sample-efficient alignment of tail-class representations. Evaluated on six long-tailed classification benchmarks, including speech (e.g., rare acoustic events, accents) and image tasks, ViRN achieves a 10.24% average accuracy gain over state-of-the-art methods.