Robust Unsupervised Learning via L-Statistic Minimization

TL;DR

This paper introduces an unsupervised learning method that enhances robustness against data distribution perturbations by minimizing an L-statistic criterion, effectively improving model stability and accuracy.

Contribution

It proposes a novel L-statistic minimization approach for robust unsupervised learning, with theoretical analysis and practical validation on clustering and subspace tasks.

Findings

The method improves robustness to data perturbations.

Theoretical bounds on reconstruction error are established.

Numerical experiments validate effectiveness on clustering and subspace analysis.

Abstract

Designing learning algorithms that are resistant to perturbations of the underlying data distribution is a problem of wide practical and theoretical importance. We present a general approach to this problem focusing on unsupervised learning. The key assumption is that the perturbing distribution is characterized by larger losses relative to a given class of admissible models. This is exploited by a general descent algorithm which minimizes an $L$-statistic criterion over the model class, weighting small losses more. Our analysis characterizes the robustness of the method in terms of bounds on the reconstruction error relative to the underlying unperturbed distribution. As a byproduct, we prove uniform convergence bounds with respect to the proposed criterion for several popular models in unsupervised learning, a result which may be of independent interest.Numerical experiments with kmeans clustering and principal subspace analysis demonstrate the effectiveness of our approach.