On a Class of Parameters Estimators in Linear Models Dominating the Least Squares one, Based on Compressed Sensing Techniques

TL;DR

This paper introduces a new parameter estimator for linear models using compressed sensing techniques that outperforms least squares under certain noise conditions, with practical advantages demonstrated through simulations.

Contribution

It proposes a novel $l_1$ norm-based estimator for extended parameters in underdetermined linear models, showing dominance over least squares when noise exceeds a threshold.

Findings

Estimator outperforms least squares in high-noise scenarios

Effective even with estimated or incomplete design matrices

Useful for solving complex inverse ill-posed problems

Abstract

The estimation of parameters in a linear model is considered under the hypothesis that the noise, with finite second order statistics, can be represented in a given deterministic basis by random coefficients. An extended underdetermined design matrix is then considered and an estimator of the extended parameters is proposed with minimum $l_1$ norm. It is proved that if the noise variance is larger than a threshold, which depends on the unknown parameters and on the extended design matrix, then the proposed estimator of the original parameters dominates the least-squares estimator in the sense of the mean square error. A small simulation illustrates the behavior of the proposed estimator. Moreover it is shown experimentally that the proposed estimator can be convenient even if the design matrix is not known but only an estimate can be used. Furthermore the noise basis can eventually be used to introduce some prior information in the estimation process. These points are illustrated by simulation by using the proposed estimator for solving a difficult inverse ill-posed problem related to the complex moments of an atomic complex measure.