Fast and Efficient Implementation of the Maximum Likelihood Estimation for the Linear Regression with Gaussian Model Uncertainty

TL;DR

This paper extends the analysis of maximum likelihood estimation in linear regression with Gaussian model uncertainty to both overdetermined and underdetermined cases, proving convexity and strong duality, and introduces a fast, unified implementation called GRV-ML.

Contribution

It generalizes the MLE analysis to rank-deficient mean measurement matrices and proposes a fast, unified algorithm for both overdetermined and underdetermined systems.

Findings

MLE problem is convex and satisfies strong duality.

Randomness in measurement matrix can be beneficial in underdetermined cases.

Proposed GRV-ML algorithm is fast and effective across different system types.

Abstract

The linear regression model with a random variable (RV) measurement matrix, where the mean of the random measurement matrix has full column rank, has been extensively studied. In particular, the quasiconvexity of the maximum likelihood estimation (MLE) problem was established, and the corresponding Cramer-Rao bound (CRB) was derived, leading to the development of an efficient bisection-based algorithm known as RV-ML. In contrast, this work extends the analysis to both overdetermined and underdetermined cases, allowing the mean of the random measurement matrix to be rank-deficient. A remarkable contribution is the proof that the equivalent MLE problem is convex and satisfies strong duality, strengthening previous quasiconvexity results. Moreover, it is shown that in underdetermined scenarios, the randomness in the measurement matrix can be beneficial for estimation under certain conditions. In addition, a fast and unified implementation of the MLE solution, referred to as generalized RV-ML (GRV-ML), is proposed, which handles a more general case including both underdetermined and overdetermined systems. Extensive numerical simulations are provided to validate the theoretical findings.