Iterative Exploration-Driven Sparse SDP Clustering via Thompson Sampling

TL;DR

This paper introduces a robust, exploration-driven sparse clustering method using SDP relaxations and Thompson sampling, effectively handling high-dimensional data with non-informative features and escaping local optima.

Contribution

It proposes a novel iterative framework combining SDP-based clustering with Thompson sampling for feature selection, enhancing robustness and exploration in high-dimensional sparse clustering.

Findings

The method achieves exact clustering under certain conditions.

It outperforms existing sparse clustering techniques in experiments.

The approach is scalable and effective with unknown covariance.

Abstract

This paper studies high-dimensional sparse clustering, a combinatorial NP-hard problem arising from the bilinear coupling between cluster assignment and feature selection. We analyze semidefinite programming (SDP) relaxations of $K$-means and establish minimax separation bounds, demonstrating that these relaxations are theoretically robust to feature over-selection: exact recovery is preserved even in the presence of non-informative features. Leveraging this robustness, we propose a block-coordinate ascent framework that alternates between SDP-based clustering and non-conservative feature selection. To address the tendency of deterministic greedy methods to become trapped in local optima, we formulate the feature selection step as a Thompson sampling bandit problem. This approach introduces adaptive memory by aggregating historical variable-selection outcomes into posterior distributions, and selects features via posterior sampling, enabling stochastic exploration that promotes the inclusion of under-explored features and facilitates escape from local maxima. We establish conditions for consistent variable selection and exact clustering recovery, and extend the method to settings with unknown covariance through a scalable, inverse-free estimation procedure. Numerical experiments demonstrate that the proposed memory-driven approach consistently outperforms state-of-the-art sparse clustering methods.