Fast Super-Paramagnetic Clustering

TL;DR

This paper introduces a fast, maximum likelihood-based clustering method for financial data that effectively uncovers hierarchical structures and sector groupings in stock markets, outperforming traditional approaches in high-dimensional settings.

Contribution

The paper presents a novel, computationally efficient variant of super-paramagnetic clustering called f-SPC, which converges to optimal solutions in high-dimensional financial datasets.

Findings

f-SPC effectively recovers economic sector clusters

Solutions converge within the super-paramagnetic phase where entropy is maximized

Method outperforms existing clustering approaches on stock market data

Abstract

We map stock market interactions to spin models to recover their hierarchical structure using a simulated annealing based Super-Paramagnetic Clustering (SPC) algorithm. This is directly compared to a modified implementation of a maximum likelihood approach we call Fast Super-Paramagnetic Clustering (f-SPC). The methods are first applied standard toy test-case problems, and then to a data-set of 447 stocks traded on the New York Stock Exchange (NYSE) over 1249 days. The signal to noise ratio of stock market correlation matrices is briefly considered. Our result recover approximately clusters representative of standard economic sectors and mixed ones whose dynamics shine light on the adaptive nature of financial markets and raise concerns relating to the effectiveness of industry based static financial market classification in the world of real-time data analytics. A key result is that we show that f-SPC maximum likelihood solutions converge to ones found within the Super-Paramagnetic Phase where the entropy is maximum, and those solutions are qualitatively better for high dimensionality data-sets.