Rockafellian Relaxation and Stochastic Optimization under Perturbations

TL;DR

This paper introduces an optimistic Rockafellian relaxation framework for stochastic optimization that handles data inaccuracies, ambiguous distributions, and outliers without requiring convexity or smoothness, demonstrated through computer vision and text analytics.

Contribution

It develops a novel, assumption-free relaxation approach that jointly optimizes over decisions and model perturbations, addressing challenges in stochastic optimization under uncertainty.

Findings

Framework handles ambiguous distributions and outliers effectively.

Convergence rates analyzed under changing distributions.

Numerical experiments demonstrate practical applicability in vision and text analytics.

Abstract

In practice, optimization models are often prone to unavoidable inaccuracies due to dubious assumptions and corrupted data. Traditionally, this placed special emphasis on risk-based and robust formulations, and their focus on ``conservative" decisions. We develop, in contrast, an ``optimistic" framework based on Rockafellian relaxations in which optimization is conducted not only over the original decision space but also jointly with a choice of model perturbation. The framework enables us to address challenging problems with ambiguous probability distributions from the areas of two-stage stochastic optimization without relatively complete recourse, probability functions lacking continuity properties, expectation constraints, and outlier analysis. We are also able to circumvent the fundamental difficulty in stochastic optimization that convergence of distributions fails to guarantee convergence of expectations. The framework centers on the novel concepts of exact and limit-exact Rockafellians, with interpretations of ``negative'' regularization emerging in certain settings. We illustrate the role of Phi-divergence, examine rates of convergence under changing distributions, and explore extensions to first-order optimality conditions. The main development is free of assumptions about convexity, smoothness, and even continuity of objective functions. Numerical results in the setting of computer vision and text analytics with label noise illustrate the framework.