A Riemannian Optimization Approach to Clustering Problems

TL;DR

This paper introduces a Riemannian optimization framework for clustering problems, unifying models like k-means and community detection, and proposes an efficient algorithm with proven convergence.

Contribution

It formulates clustering as a Riemannian optimization problem on a specific manifold and develops an inexact accelerated proximal gradient method with convergence guarantees.

Findings

Effective in community detection tasks

Competitive performance in image segmentation

Theoretical convergence guarantees

Abstract

This paper considers the optimization problem in the form of $\min_{X \in \mathcal{F}_v} f(x) + \lambda \|X\|_1,$ where $f$ is smooth, $\mathcal{F}_v = \{X \in \mathbb{R}^{n \times q} : X^T X = I_q, v \in \mathrm{span}(X)\}$, and $v$ is a given positive vector. The clustering models including but not limited to the models used by $k$-means, community detection, and normalized cut can be reformulated as such optimization problems. It is proven that the domain $\mathcal{F}_v$ forms a compact embedded submanifold of $\mathbb{R}^{n \times q}$ and optimization-related tools including a family of computationally efficient retractions and an orthonormal basis of any normal space of $\mathcal{F}_v$ are derived. An inexact accelerated Riemannian proximal gradient method that allows adaptive step size is proposed and its global convergence is established. Numerical experiments on community detection in networks and normalized cut for image segmentation are used to demonstrate the performance of the proposed method.