On proportional volume sampling for experimental design in general spaces

TL;DR

This paper extends proportional volume sampling (PVS) to general spaces, showing it preserves optimality guarantees, can be sampled efficiently, and is useful in complex design scenarios, despite some practical limitations.

Contribution

It generalizes PVS for arbitrary Polish spaces, proves preservation of A- and D-optimality guarantees, and explores PVS as a practical heuristic tool.

Findings

PVS preserves A- and D-optimality guarantees in general spaces.

PVS can be sampled in polynomial time.

PVS is effective for complex, low-dimensional design spaces with irregular shapes.

Abstract

Optimal design for linear regression is a fundamental task in statistics. For finite design spaces, recent progress has shown that random designs drawn using proportional volume sampling (PVS) lead to approximation guarantees for A-optimal design. PVS strikes the balance between design nodes that jointly fill the design space, while marginally staying in regions of high mass under the solution of a relaxed convex version of the original problem. In this paper, we examine some of the statistical implications of a new variant of PVS for (possibly Bayesian) optimal design. Using point process machinery, we treat the case of a generic Polish design space. We show that not only are the A-optimality approximation guarantees preserved, but we obtain similar guarantees for D-optimal design that tighten recent results. Moreover, we show that PVS can be sampled in polynomial time. Unfortunately, in spite of its elegance and tractability, we demonstrate on a simple example that the practical implications of general PVS are likely limited. In the second part of the paper, we focus on applications and investigate the use of PVS as a subroutine for stochastic search heuristics. We demonstrate that PVS is a robust addition to the practitioner's toolbox, especially when the regression functions are nonstandard and the design space, while low-dimensional, has a complicated shape (e.g., nonlinear boundaries, several connected components).