Linear regression with known noise distribution up to a scale: The reward of not using the OLSE

TL;DR

This paper demonstrates that in linear regression models with known noise distribution up to a scale, the maximum likelihood estimator can outperform the ordinary least squares estimator in terms of efficiency, especially when the noise is non-Gaussian.

Contribution

It provides a theoretical analysis of the asymptotic efficiency of the MLE over OLSE for scale family noise distributions and quantifies the efficiency gain based on deviation from Gaussianity.

Findings

MLE outperforms OLSE for non-Gaussian noise distributions.

Efficiency gain depends on the deviation from Gaussian distribution.

Simulation and real data illustrate the theoretical results.

Abstract

While the ordinary least squares estimator (OLSE) is still the most used estimator in linear regression models, other estimators can be more efficient when the error distribution is not Gaussian. In this paper, our goal is to evaluate this efficiency in the case of the Maximum Likelihood estimator (MLE) when the noise distribution belongs to a scale family. Under some regularity conditions, we show that (\beta_n,s_n), the MLE of the unknown regression vector \beta_0 and the scale s_0 exists and give the expression of the asymptotic efficiency of \beta_n over the OLSE. For given three scale families of densities, we quantify the true statistical gain of the MLE as a function of their deviation from the Gaussian family. To illustrate the theory, we present simulation results for different settings and also compare the MLE to the OLSE for the real market fish dataset.