Group Symmetry and Covariance Regularization

TL;DR

This paper introduces a group invariance framework for regularizing covariance matrices in high-dimensional models with symmetry, leading to improved sample efficiency and insights into model selection.

Contribution

It formalizes symmetry via group invariance and proposes projection onto fixed point subspaces as a novel regularization method for covariance matrices.

Findings

Derived precise convergence rates for regularized covariance matrices.

Demonstrated statistical gains in sample complexity due to symmetry.

Validated results through simulations.

Abstract

Statistical models that possess symmetry arise in diverse settings such as random fields associated to geophysical phenomena, exchangeable processes in Bayesian statistics, and cyclostationary processes in engineering. We formalize the notion of a symmetric model via group invariance. We propose projection onto a group fixed point subspace as a fundamental way of regularizing covariance matrices in the high-dimensional regime. In terms of parameters associated to the group we derive precise rates of convergence of the regularized covariance matrix and demonstrate that significant statistical gains may be expected in terms of the sample complexity. We further explore the consequences of symmetry on related model-selection problems such as the learning of sparse covariance and inverse covariance matrices. We also verify our results with simulations.