Task Agnostic Robust Learning on Corrupt Outputs by Correlation-Guided Mixture Density Networks

TL;DR

This paper introduces a correlation-guided mixture density network approach for robust weakly supervised learning on noisy data, effectively estimating target distributions and data quality in classification and regression tasks.

Contribution

It proposes a novel differentiable model with a Cholesky Block to handle dependencies among mixture distributions, improving robustness to noisy outputs.

Findings

Consistently outperforms baseline methods on noisy data tasks.

Effectively estimates data quality and target distribution simultaneously.

Demonstrates versatility in both classification and regression problems.

Abstract

In this paper, we focus on weakly supervised learning with noisy training data for both classification and regression problems.We assume that the training outputs are collected from a mixture of a target and correlated noise distributions.Our proposed method simultaneously estimates the target distribution and the quality of each data which is defined as the correlation between the target and data generating distributions.The cornerstone of the proposed method is a Cholesky Block that enables modeling dependencies among mixture distributions in a differentiable manner where we maintain the distribution over the network weights.We first provide illustrative examples in both regression and classification tasks to show the effectiveness of the proposed method.Then, the proposed method is extensively evaluated in a number of experiments where we show that it constantly shows comparable or superior performances compared to existing baseline methods in the handling of noisy data.