Inference From Random Restarts

TL;DR

This paper develops a probabilistic framework to interpret repeated solutions from random restarts in nonconvex optimization, providing conditions for finite outcomes and Bayesian inference of solution uniqueness.

Contribution

It introduces a formal probabilistic approach to analyze the effectiveness of random restarts in nonconvex problems, including conditions for finite outcomes and Bayesian inference methods.

Findings

Standard algorithms satisfy the conditions for finite outcomes.

Repeated runs can be modeled as independent samples from outcome distributions.

Framework clarifies when repeated solutions indicate uniqueness.

Abstract

Algorithms for computing equilibria, optima, and fixed points in nonconvex problems often depend sensitively on practitioner-chosen initial conditions. When uniqueness of a solution is of interest, a common heuristic is to run such algorithms from many randomly selected initial conditions and to interpret repeated convergence to the same output as evidence of a unique solution or a dominant basin of attraction. Despite its widespread use, this practice lacks a formal inferential foundation. We provide a simple probabilistic framework for interpreting such numerical evidence. First, we give sufficient conditions under which an algorithm's terminal output is a measurable function of its initial condition, allowing probabilistic reasoning over outcomes. Second, we provide sufficient conditions ensuring that an algorithm admits only finitely many possible terminal outcomes. While these conditions may be difficult to verify on a case-by-case basis, we give simple sufficient conditions for broad classes of problems under which almost all instances admit only finitely many outcomes (in the sense of prevalence). Standard algorithms such as gradient descent and damped fixed-point iteration applied to sufficiently smooth functions satisfy these conditions. Within this framework, repeated solver runs correspond to independent samples from the induced distribution over outcomes. We adopt a Bayesian approach to infer basin sizes and the probability of solution uniqueness from repeated identical outputs, and we establish convergence rates for the resulting posterior beliefs. Finally, we apply our framework to settings in the existing industrial organization literature, where random-restart heuristics are used. Our results formalize and qualify these arguments, clarifying when repeated convergence provides meaningful evidence for uniqueness and when it does not.