Bootstraps Regularize Singular Correlation Matrices

TL;DR

This paper demonstrates that averaging multiple bootstrapped correlation matrices can effectively regularize singular matrices, ensuring positive-definiteness especially when the number of bootstraps exceeds a certain threshold related to data dimensions.

Contribution

It provides an analytical proof that bootstrapping can rapidly regularize singular correlation matrices, with explicit bounds on the number of bootstraps needed for positive-definiteness.

Findings

Average of k bootstrapped correlation matrices becomes positive-definite as k increases.

The required number of bootstraps is less than the number of objects n.

Method is applicable in high-dimensional data fields like finance and genetics.

Abstract

I show analytically that the average of $k$ bootstrapped correlation matrices rapidly becomes positive-definite as $k$ increases, which provides a simple approach to regularize singular Pearson correlation matrices. If $n$ is the number of objects and $t$ the number of features, the averaged correlation matrix is almost surely positive-definite if $k> \frac{e}{e-1}\frac{n}{t}\simeq 1.58\frac{n}{t}$ in the limit of large $t$ and $n$. The probability of obtaining a positive-definite correlation matrix with $k$ bootstraps is also derived for finite $n$ and $t$. Finally, I demonstrate that the number of required bootstraps is always smaller than $n$. This method is particularly relevant in fields where $n$ is orders of magnitude larger than the size of data points $t$, e.g., in finance, genetics, social science, or image processing.