Learning with Correntropy-induced Losses for Regression with Mixture of Symmetric Stable Noise

TL;DR

This paper investigates correntropy-based regression under non-Gaussian noise modeled as a mixture of symmetric stable distributions, establishing its effectiveness and optimal learning rates in robust regression scenarios.

Contribution

It introduces a theoretical framework for correntropy regression with mixture of symmetric stable noise, demonstrating asymptotic optimal learning rates without requiring finite variance.

Findings

Correntropy regression can effectively learn the conditional mean and median functions.

Established asymptotic learning rates of order O(n^{-1}) for the estimators.

Results justify the robustness of correntropy-based methods against outliers and non-Gaussian noise.

Abstract

In recent years, correntropy and its applications in machine learning have been drawing continuous attention owing to its merits in dealing with non-Gaussian noise and outliers. However, theoretical understanding of correntropy, especially in the statistical learning context, is still limited. In this study, within the statistical learning framework, we investigate correntropy based regression in the presence of non-Gaussian noise or outliers. Motivated by the practical way of generating non-Gaussian noise or outliers, we introduce mixture of symmetric stable noise, which include Gaussian noise, Cauchy noise, and their mixture as special cases, to model non-Gaussian noise or outliers. We demonstrate that under the mixture of symmetric stable noise assumption, correntropy based regression can learn the conditional mean function or the conditional median function well without resorting to the finite-variance or even the finite first-order moment condition on the noise. In particular, for the above two cases, we establish asymptotic optimal learning rates for correntropy based regression estimators that are asymptotically of type $\mathcal{O}(n^{-1})$. These results justify the effectiveness of the correntropy based regression estimators in dealing with outliers as well as non-Gaussian noise. We believe that the present study completes our understanding towards correntropy based regression from a statistical learning viewpoint, and may also shed some light on robust statistical learning for regression.