Ratio-consistent estimation for long range dependent Toeplitz covariance with application to matrix data whitening

TL;DR

This paper investigates the ratio consistency of Toeplitz covariance estimators under long-range dependence, proposing a new whitening method with applications in signal detection and PCA-based noise reduction.

Contribution

It introduces ratio and ratio LSD consistency results for Toeplitz covariance estimators in LRD settings, enabling effective data whitening and analysis.

Findings

Ratio consistency established for the unbiased estimator under LRD.

Weaker ratio LSD consistency shown for the biased estimator.

Application of the whitening procedure to signal detection and PCA.

Abstract

We consider a data matrix $X:=C_N^{1/2}ZR_M^{1/2}$ from a multivariate stationary process with a separable covariance function, where $C_N$ is a $N\times N$ positive semi-definite matrix, $Z$ a $N\times M$ random matrix of uncorrelated standardized white noise, and $R_M$ a $M\times M$ Toeplitz matrix. Under the assumption of long range dependence (LRD), we re-examine the consistency of two toeplitzifized estimators $\hat R_M$ (unbiased) and $\hat R_M^b$ (biased) for $R_M$, which are known to be norm consistent with $R_M$ when the process is short range dependent (SRD). However in the LRD case, some simulations suggest that the norm consistency does not hold in general for both estimators. Instead, a weaker {\it ratio consistency} is established for the unbiased estimator $\hat R_M$, and a further weaker {\it ratio LSD consistency} is established for the biased estimator $\hat R_M^b$. The main result leads to a consistent whitening procedure on the original data matrix $X$, which is further applied to two real world questions, one is a signal detection problem, and the other is PCA on the space covariance $C_N$ to achieve a noise reduction and data compression.