Gradient flow for finding E-optimal designs

TL;DR

This paper introduces a Wasserstein gradient flow approach for E-optimal design, addressing nonsmooth eigenvalue criteria and demonstrating improved reliability over traditional methods in high-dimensional models.

Contribution

It develops a novel constrained Wasserstein steepest-ascent method for nonsmooth optimal design criteria, with theoretical guarantees and practical advantages.

Findings

The Wasserstein gradient at empirical measures aligns with Euclidean gradients for smooth criteria.

The proposed ascent method achieves near-optimal E-criterion values in numerical experiments.

It outperforms particle swarm optimization in higher-dimensional settings.

Abstract

The $E$-optimality criterion for a regression model maximizes the smallest eigenvalue of the information matrix and becomes non-differentiable when this eigenvalue has multiplicity greater than one. Working in the $2$-Wasserstein space, we show that the Wasserstein gradient at an empirical measure coincides, up to a constant factor, with the Euclidean particle gradient for smooth criteria such as $D$- and $L$-optimality, and that the approximation gap for equal-weight $N$-particle designs vanishes at an explicit rate. The main challenge is the nonsmooth $E$-criterion, for which the Wasserstein gradient does not exist. We replace it with a constrained Wasserstein steepest-ascent field obtained by maximizing feasible directional derivatives over the tangent cone of the design space, and prove that the resulting flow satisfies an exact energy identity and that every limit point is first-order stationary. The particle ascent computation reduces to a convex semidefinite programme whose dimension equals the multiplicity of the smallest eigenvalue. In numerical comparisons on second-order response surface models and a seven-dimensional logistic regression model, the constrained Wasserstein steepest-ascent method attains near-optimal $E$-criterion values and is markedly more reliable than particle swarm optimization in higher-dimensional settings. The framework applies more broadly to other nonsmooth minimax criteria in optimal design, and a numerical experiment on the minimax-single-parameter criterion confirms that the method attains the theoretical optimum.