A Geometric Approach to $k$-means

TL;DR

This paper introduces a geometric framework for -means clustering that improves the ability to find global optima by escaping local solutions through iterative detection and correction of mis-specified clusters.

Contribution

It presents a novel geometric algorithmic framework that unifies and enhances existing -means methods by systematically escaping local optima and handling over- or under-specified cluster counts.

Findings

Framework effectively escapes local optima in -means clustering.

Theoretical guarantees support the framework's convergence to global solutions.

Extensive experiments validate the approach's superior performance.

Abstract

\kmeans clustering is a fundamental problem in many scientific and engineering domains. The optimization problem associated with \kmeans clustering is nonconvex, for which standard algorithms are only guaranteed to find a local optimum. Leveraging the hidden structure of local solutions, we propose a general algorithmic framework for escaping undesirable local solutions and recovering the global solution or the ground truth clustering. This framework consists of iteratively alternating between two steps: (i) detect mis-specified clusters in a local solution, and (ii) improve the local solution by non-local operations. We discuss specific implementation of these steps, and elucidate how the proposed framework unifies many existing variants of \kmeans algorithms through a geometric perspective. We also present two natural variants of the proposed framework, where the initial number of clusters may be over- or under-specified. We provide theoretical justifications and extensive experiments to demonstrate the efficacy of the proposed approach.