Spherical clustering in detection of groups of concomitant extremes

TL;DR

This paper investigates spherical clustering methods for detecting groups of concomitant extremes in high-dimensional data, providing theoretical support, identifying pitfalls, and proposing a new clustering algorithm that outperforms traditional methods in challenging scenarios.

Contribution

It offers one of the first theoretical justifications for spherical $k$-means in this context and introduces a novel spherical $k$-principal-components clustering algorithm with proven success conditions.

Findings

Spherical $k$-means performs well empirically for concomitant extremes.

An alternative cost function improves clustering accuracy.

The $k$-principal-components method outperforms $k$-means in weak dependence scenarios.

Abstract

There is growing empirical evidence that spherical $k$-means clustering performs well at identifying groups of concomitant extremes in high dimensions, thereby leading to sparse models. We provide one of the first theoretical results supporting this approach, but also demonstrate some pitfalls. Furthermore, we show that an alternative cost function may be more appropriate for identifying concomitant extremes, and it results in a novel spherical $k$-principal-components clustering algorithm. Our main result establishes a broadly satisfied sufficient condition guaranteeing the success of this method, albeit in a rather basic setting. Finally, we illustrate in simulations that $k$-principal-components outperforms $k$-means in the difficult case of weak asymptotic dependence within the groups.