Joint Optimization and Statistical Inference for Zero-th Order Simulation Optimization

TL;DR

This paper introduces a new stochastic optimization algorithm that achieves fast convergence while enabling valid statistical inference through a central limit theorem with vanishing bias, addressing limitations of classical methods.

Contribution

The paper proposes a novel joint optimization and inference algorithm that maintains rapid convergence and ensures asymptotically valid confidence intervals in stochastic optimization.

Findings

The new algorithm achieves fast convergence similar to classical methods.

It allows for the construction of valid confidence intervals with a CLT with vanishing bias.

Numerical experiments confirm the theoretical advantages of the proposed method.

Abstract

We consider stochastic optimization problems with the dual tasks of (i) effectively finding the optimizer and (ii) reliably conducting statistical inference for the optimal objective function value. We find that classical simulation optimization and stochastic optimization algorithms, despite of their fast convergence rates to the optimizer under strong convexity assumptions, may not come with a valid central limit theorem (CLT) with a vanishing bias. This non-vanishing bias can harm statistical inference and the construction of asymptotically valid confidence intervals. We fix this issue by providing a new stochastic optimization algorithm that on one hand maintains the same fast convergence rate and on the other hand permits the establishment of a valid CLT with vanishing bias. We discuss practical implementations of the proposed algorithm and conduct numerical experiments to illustrate the theoretical findings.