On optimal designs for non-regular models

TL;DR

This paper introduces Hellinger information as a new measure to define and analyze optimal experimental designs for non-regular models where Fisher information is not applicable, providing theoretical bounds and practical design solutions.

Contribution

It proposes Hellinger information as a generalization of Fisher information for non-regular models and derives optimal designs based on this new measure.

Findings

Hellinger information bounds the minimax risk in non-regular models.

Hellinger optimal designs outperform traditional designs in non-regular regression problems.

Numerical results confirm the efficiency of Hellinger-based designs.

Abstract

Classically, Fisher information is the relevant object in defining optimal experimental designs. However, for models that lack certain regularity, the Fisher information does not exist and, hence, there is no notion of design optimality available in the literature. This article seeks to fill the gap by proposing a so-called Hellinger information, which generalizes Fisher information in the sense that the two measures agree in regular problems, but the former also exists for certain types of non-regular problems. We derive a Hellinger information inequality, showing that Hellinger information defines a lower bound on the local minimax risk of estimators. This provides a connection between features of the underlying model---in particular, the design---and the performance of estimators, motivating the use of this new Hellinger information for non-regular optimal design problems. Hellinger optimal designs are derived for several non-regular regression problems, with numerical results empirically demonstrating the efficiency of these designs compared to alternatives.