Neural Prior Estimation: Learning Class Priors from Latent Representations

TL;DR

This paper introduces NPE, a neural framework that learns class priors from latent features to improve predictions in imbalanced datasets, with theoretical guarantees and practical effectiveness.

Contribution

The paper proposes NPE, a novel method for learning class priors directly from latent representations, enhancing bias correction without explicit class count reliance.

Findings

NPE accurately recovers class priors under Neural Collapse.

NPE-LA improves performance on long-tailed classification tasks.

Experiments show consistent gains on CIFAR and segmentation benchmarks.

Abstract

Class imbalance induces systematic bias in deep neural networks by imposing a skewed effective class prior. This work introduces the Neural Prior Estimator (NPE), a framework that learns feature-conditioned log-prior estimates from latent representations. NPE employs one or more Prior Estimation Modules trained jointly with the backbone via a one-way logistic loss. Under the Neural Collapse regime, NPE is analytically shown to recover the class log-prior up to an additive constant, providing a theoretically grounded adaptive signal without requiring explicit class counts or distribution-specific hyperparameters. The learned estimate is incorporated into logit adjustment, forming NPE-LA, a principled mechanism for bias-aware prediction. Experiments on long-tailed CIFAR and imbalanced semantic segmentation benchmarks (STARE, ADE20K) demonstrate consistent improvements, particularly for underrepresented classes. NPE thus offers a lightweight and theoretically justified approach to learned prior estimation and imbalance-aware prediction.