Sequential Inverse Approximation of a Regularized Sample Covariance Matrix

TL;DR

This paper introduces sequential update rules for approximating the inverse of a regularized sample covariance matrix, facilitating scalable and efficient high-dimensional data analysis in machine learning.

Contribution

It presents novel sequential algorithms for inverse covariance matrix approximation using shrinkage estimators, enhancing large-scale machine learning methods.

Findings

Derived sequential update rules for inverse covariance approximation.

Improves scalability of covariance-based methods in high-dimensional settings.

Enables efficient sequential learning with regularized covariance matrices.

Abstract

One of the goals in scaling sequential machine learning methods pertains to dealing with high-dimensional data spaces. A key related challenge is that many methods heavily depend on obtaining the inverse covariance matrix of the data. It is well known that covariance matrix estimation is problematic when the number of observations is relatively small compared to the number of variables. A common way to tackle this problem is through the use of a shrinkage estimator that offers a compromise between the sample covariance matrix and a well-conditioned matrix, with the aim of minimizing the mean-squared error. We derived sequential update rules to approximate the inverse shrinkage estimator of the covariance matrix. The approach paves the way for improved large-scale machine learning methods that involve sequential updates.