Stochastic Trace Optimization of Parameter Dependent Matrices Based on Statistical Learning Theory

TL;DR

This paper introduces a Monte Carlo method for optimizing the trace of parameter-dependent matrices, providing probabilistic bounds on the estimator's error using epsilon nets and generic chaining, with implications for matrices with small off-diagonal mass.

Contribution

It develops a stochastic trace optimization technique with theoretical error bounds based on advanced probabilistic tools, applicable to nonlinear parameter-dependent matrices.

Findings

Bounds are tight for matrices with small off-diagonal mass.

Epsilon net bounds are easier to compute with explicit constants.

Chaining bounds may outperform epsilon net bounds in some cases.

Abstract

We consider matrices $\boldsymbol{A}(\boldsymbol\theta)\in\mathbb{R}^{m\times m}$ that depend, possibly nonlinearly, on a parameter $\boldsymbol\theta$ from a compact parameter space $\Theta$. We present a Monte Carlo estimator for minimizing $\text{trace}(\boldsymbol{A}(\boldsymbol\theta))$ over all $\boldsymbol\theta\in\Theta$, and determine the sampling amount so that the backward error of the estimator is bounded with high probability. We derive two types of bounds, based on epsilon nets and on generic chaining. Both types predict a small sampling amount for matrices $\boldsymbol{A}(\boldsymbol\theta)$ with small offdiagonal mass, and parameter spaces $\Theta$ of small ``size.'' Dependence on the matrix dimension~$m$ is only weak or not explicit. The bounds based on epsilon nets are easier to evaluate and come with fully specified constants. In contrast, the bounds based on chaining depend on the Talagrand functionals which are difficult to evaluate, except in very special cases. Comparisons between the two types of bounds are difficult, although the literature suggests that chaining bounds can be superior.