A Note on Parameter Estimation for Misspecified Regression Models with Heteroskedastic Errors

TL;DR

This paper investigates how to optimally estimate parameters in misspecified linear regression models with heteroskedastic errors, revealing that standard weighting can be counterproductive and proposing an adaptive estimator with improved asymptotic properties.

Contribution

It introduces an adaptive estimator for misspecified regression models with heteroskedastic errors that outperforms traditional weighted least squares in terms of asymptotic variance.

Findings

Weighted least squares can be suboptimal for non-linear responses.

The proposed adaptive estimator has lower asymptotic variance.

Simulation and application demonstrate estimator's effectiveness.

Abstract

Misspecified models often provide useful information about the true data generating distribution. For example, if $y$ is a non-linear function of $x$ the least squares estimator $\hat{\beta}$ is an estimate of $\beta$, the slope of the best linear approximation to the non-linear function. Motivated by problems in astronomy, we study how to incorporate observation measurement error variances into fitting parameters of misspecified models. Our asymptotic theory focuses on the particular case of linear regression where often weighted least squares procedures are used to account for heteroskedasticity. We find that when the response is a non-linear function of the independent variable, the standard procedure of weighting by the inverse of the observation variances can be counter-productive. In particular, ordinary least squares may have lower asymptotic variance. We construct an adaptive estimator which has lower asymptotic variance than either OLS or standard WLS. We demonstrate our theory in a small simulation and apply these ideas to the problem of estimating the period of a periodic function using a sinusoidal model.