Model Mismatch Trade-offs in LMMSE Estimation

TL;DR

This paper analyzes how model mismatch affects the performance of LMMSE estimators, revealing trade-offs between sample size, model complexity, and estimation accuracy in linear systems.

Contribution

It introduces a framework modeling regressors as random variables to quantify the impact of model mismatch on MSE in LMMSE estimation.

Findings

MSE depends on sample size and model parameters

Increasing samples or model complexity alone may not improve performance

Performance degradation occurs when sample size is insufficient

Abstract

We consider a linear minimum mean squared error (LMMSE) estimation framework with model mismatch where the assumed model order is smaller than that of the underlying linear system which generates the data used in the estimation process. By modelling the regressors of the underlying system as random variables, we analyze the average behaviour of the mean squared error (MSE). Our results quantify how the MSE depends on the interplay between the number of samples and the number of parameters in the underlying system and in the assumed model. In particular, if the number of samples is not sufficiently large, neither increasing the number of samples nor the assumed model complexity is sufficient to guarantee a performance improvement.