Inference From Random Restarts
Moeen Nehzati, Diego Cussen

TL;DR
This paper develops a probabilistic framework to interpret repeated solutions from random restarts in nonconvex optimization, providing conditions for finite outcomes and Bayesian inference of solution uniqueness.
Contribution
It introduces a formal probabilistic approach to analyze the effectiveness of random restarts in nonconvex problems, including conditions for finite outcomes and Bayesian inference methods.
Findings
Standard algorithms satisfy the conditions for finite outcomes.
Repeated runs can be modeled as independent samples from outcome distributions.
Framework clarifies when repeated solutions indicate uniqueness.
Abstract
Algorithms for computing equilibria, optima, and fixed points in nonconvex problems often depend sensitively on practitioner-chosen initial conditions. When uniqueness of a solution is of interest, a common heuristic is to run such algorithms from many randomly selected initial conditions and to interpret repeated convergence to the same output as evidence of a unique solution or a dominant basin of attraction. Despite its widespread use, this practice lacks a formal inferential foundation. We provide a simple probabilistic framework for interpreting such numerical evidence. First, we give sufficient conditions under which an algorithm's terminal output is a measurable function of its initial condition, allowing probabilistic reasoning over outcomes. Second, we provide sufficient conditions ensuring that an algorithm admits only finitely many possible terminal outcomes. While these…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStochastic Gradient Optimization Techniques · Risk and Portfolio Optimization · Gaussian Processes and Bayesian Inference
