Adaptive Stochastic Gradient Descent on the Grassmannian for Robust Low-Rank Subspace Recovery and Clustering

TL;DR

GASG21 is an adaptive stochastic gradient algorithm on the Grassmannian for robust low-rank subspace recovery and clustering, effectively handling heavy column outliers with improved convergence.

Contribution

It introduces a novel adaptive step-size stochastic gradient method on the Grassmannian for robust low-rank subspace recovery and clustering, with demonstrated efficiency.

Findings

Effective recovery of low-rank subspaces with heavy outliers

Robust clustering of corrupted data into subspaces

Accelerated convergence through adaptive step-size tuning

Abstract

In this paper, we present GASG21 (Grassmannian Adaptive Stochastic Gradient for $L_{2,1}$ norm minimization), an adaptive stochastic gradient algorithm to robustly recover the low-rank subspace from a large matrix. In the presence of column outliers, we reformulate the batch mode matrix $L_{2,1}$ norm minimization with rank constraint problem as a stochastic optimization approach constrained on Grassmann manifold. For each observed data vector, the low-rank subspace $\mathcal{S}$ is updated by taking a gradient step along the geodesic of Grassmannian. In order to accelerate the convergence rate of the stochastic gradient method, we choose to adaptively tune the constant step-size by leveraging the consecutive gradients. Furthermore, we demonstrate that with proper initialization, the K-subspaces extension, K-GASG21, can robustly cluster a large number of corrupted data vectors into a union of subspaces. Numerical experiments on synthetic and real data demonstrate the efficiency and accuracy of the proposed algorithms even with heavy column outliers corruption.