Label Distribution Learning via Implicit Distribution Representation

TL;DR

This paper introduces an implicit distribution approach using deep learning and Gaussian priors to better model label uncertainty in label distribution learning, improving robustness against noisy data.

Contribution

It proposes a novel implicit distribution representation with Gaussian priors and self-attention for label distribution learning, addressing noise and uncertainty issues.

Findings

Effective noise mitigation in label distribution datasets

Improved model robustness and accuracy

Enhanced representation of label uncertainty

Abstract

In contrast to multi-label learning, label distribution learning characterizes the polysemy of examples by a label distribution to represent richer semantics. In the learning process of label distribution, the training data is collected mainly by manual annotation or label enhancement algorithms to generate label distribution. Unfortunately, the complexity of the manual annotation task or the inaccuracy of the label enhancement algorithm leads to noise and uncertainty in the label distribution training set. To alleviate this problem, we introduce the implicit distribution in the label distribution learning framework to characterize the uncertainty of each label value. Specifically, we use deep implicit representation learning to construct a label distribution matrix with Gaussian prior constraints, where each row component corresponds to the distribution estimate of each label value, and this row component is constrained by a prior Gaussian distribution to moderate the noise and uncertainty interference of the label distribution dataset. Finally, each row component of the label distribution matrix is transformed into a standard label distribution form by using the self-attention algorithm. In addition, some approaches with regularization characteristics are conducted in the training phase to improve the performance of the model.