Optimal minimax random designs for weighted least squares estimators

TL;DR

This paper develops an optimal random design strategy for weighted least squares estimators in noisy function estimation, demonstrating improved performance over deterministic designs, especially for polynomial models.

Contribution

It introduces a novel asymptotic minimax criterion for random designs in weighted least squares and constructs optimal designs that outperform deterministic ones.

Findings

Optimal random minimax design differs from deterministic design.

Simulation shows better performance for quadratic and cubic functions.

Designs converge when noise variance becomes very large.

Abstract

This work studies an experimental design problem where {the values of a predictor variable, denoted by $x$}, are to be determined with the goal of estimating a function $m(x)$, which is observed with noise. A linear model is fitted to $m(x)$ but it is not assumed that the model is correctly specified. It follows that the quantity of interest is the best linear approximation of $m(x)$, which is denoted by $\ell(x)$. It is shown that in this framework the ordinary least squares estimator typically leads to an inconsistent estimation of $\ell(x)$, and rather weighted least squares should be considered. An asymptotic minimax criterion is formulated for this estimator, and a design that minimizes the criterion is constructed. An important feature of this problem is that the $x$'s should be random, rather than fixed. Otherwise, the minimax risk is infinite. It is shown that the optimal random minimax design is different from its deterministic counterpart, which was studied previously, and a simulation study indicates that it generally performs better when $m(x)$ is a quadratic or a cubic function. Another finding is that when the variance of the noise goes to infinity, the random and deterministic minimax designs coincide. The results are illustrated for polynomial regression models and the general case is also discussed.