Elucidating Robust Learning with Uncertainty-Aware Corruption Pattern Estimation

TL;DR

This paper introduces a robust learning approach that simultaneously learns clean data distributions and estimates underlying noise patterns by leveraging a mixture-of-experts model that distinguishes between aleatoric and epistemic uncertainties.

Contribution

The method uniquely combines uncertainty estimation with corruption pattern detection, advancing robust learning from noisy datasets across multiple domains.

Findings

Effective in learning clean distributions from noisy data

Accurately estimates underlying corruption patterns

Demonstrates robustness in vision and NLP tasks

Abstract

Robust learning methods aim to learn a clean target distribution from noisy and corrupted training data where a specific corruption pattern is often assumed a priori. Our proposed method can not only successfully learn the clean target distribution from a dirty dataset but also can estimate the underlying noise pattern. To this end, we leverage a mixture-of-experts model that can distinguish two different types of predictive uncertainty, aleatoric and epistemic uncertainty. We show that the ability to estimate the uncertainty plays a significant role in elucidating the corruption patterns as these two objectives are tightly intertwined. We also present a novel validation scheme for evaluating the performance of the corruption pattern estimation. Our proposed method is extensively assessed in terms of both robustness and corruption pattern estimation through a number of domains, including computer vision and natural language processing.