Modeling high dimensional point clouds with the spherical cluster model

TL;DR

This paper introduces the spherical cluster model (SC) for high-dimensional point clouds, providing a convex optimization approach and demonstrating its efficiency and median-like behavior in diverse datasets.

Contribution

The paper presents an exact solver for the SC model, analyzes its optimization properties, and evaluates its performance on high-dimensional data, advancing geometric clustering methods.

Findings

Exact solver outperforms heuristics in small to medium dimensions.

SC center acts as a high-dimensional median.

Efficient in high dimensions regardless of parameter ta.

Abstract

A parametric cluster model is a statistical model providing geometric insights onto the points defining a cluster. The {\em spherical cluster model} (SC) approximates a finite point set $P\subset \mathbb{R}^d$ by a sphere $S(c,r)$ as follows. Taking $r$ as a fraction $\eta\in(0,1)$ (hyper-parameter) of the std deviation of distances between the center $c$ and the data points, the cost of the SC model is the sum over all data points lying outside the sphere $S$ of their power distance with respect to $S$. The center $c$ of the SC model is the point minimizing this cost. Note that $\eta=0$ yields the celebrated center of mass used in KMeans clustering. We make three contributions. First, we show fitting a spherical cluster yields a strictly convex but not smooth combinatorial optimization problem. Second, we present an exact solver using the Clarke gradient on a suitable stratified cell complex defined from an arrangement of hyper-spheres. Finally, we present experiments on a variety of datasets ranging in dimension from $d=9$ to $d=10,000$, with two main observations. First, the exact algorithm is orders of magnitude faster than BFGS based heuristics for datasets of small/intermediate dimension and small values of $\eta$, and for high dimensional datasets (say $d>100$) whatever the value of $\eta$. Second, the center of the SC model behave as a parameterized high-dimensional median. The SC model is of direct interest for high dimensional multivariate data analysis, and the application to the design of mixtures of SC will be reported in a companion paper.