Covariance Eigenvector Sparsity for Compression and Denoising

TL;DR

This paper introduces a sparsity-aware covariance eigenvector estimation method for improved compression and denoising, leveraging norm-one regularization to enhance reconstruction performance even with noisy training data.

Contribution

A novel sparsity-aware linear dimensionality reduction scheme that exploits eigenvector sparsity for noise-resilient covariance estimation and improved signal reconstruction.

Findings

Enhanced reconstruction quality over existing methods.

Effective support identification of eigenvectors with low noise.

Asymptotic normality of the estimator proven.

Abstract

Sparsity in the eigenvectors of signal covariance matrices is exploited in this paper for compression and denoising. Dimensionality reduction (DR) and quantization modules present in many practical compression schemes such as transform codecs, are designed to capitalize on this form of sparsity and achieve improved reconstruction performance compared to existing sparsity-agnostic codecs. Using training data that may be noisy a novel sparsity-aware linear DR scheme is developed to fully exploit sparsity in the covariance eigenvectors and form noise-resilient estimates of the principal covariance eigenbasis. Sparsity is effected via norm-one regularization, and the associated minimization problems are solved using computationally efficient coordinate descent iterations. The resulting eigenspace estimator is shown capable of identifying a subset of the unknown support of the eigenspace basis vectors even when the observation noise covariance matrix is unknown, as long as the noise power is sufficiently low. It is proved that the sparsity-aware estimator is asymptotically normal, and the probability to correctly identify the signal subspace basis support approaches one, as the number of training data grows large. Simulations using synthetic data and images, corroborate that the proposed algorithms achieve improved reconstruction quality relative to alternatives.