Uncertainty quantification for subgradient descent, with applications to relaxations of discrete problems

TL;DR

This paper introduces an efficient method for uncertainty quantification in convex optimization with uncertain parameters, leveraging chaos expansion and subgradient methods, applicable to discrete problems like graph cuts.

Contribution

It proposes a novel approach combining chaos expansion with restarted subgradient methods to compute and analyze the statistics of optimal solutions under uncertainty.

Findings

Established convergence rates for the proposed method.

Demonstrated the approach's effectiveness on discrete problems.

Provided a way to handle non-trivial projections in optimization.

Abstract

We consider the problem of minimizing a convex function that depends on an uncertain parameter $\theta$. The uncertainty in the objective function means that the optimum, $x^*(\theta)$, is also a function of $\theta$. We propose an efficient method to compute $x^*(\theta)$ and its statistics. We use a chaos expansion of $x^*(\theta)$ along a truncated basis and study a restarted subgradient method that compute the optimal coefficients. We establish the convergence rate of the method as the number of basis functions increases, and hence the dimensionality of the optimization problem is increased. We give a non-asymptotic convergence rate for subgradient descent, building on earlier work that looked at gradient and accelerated gradient descent. Additionally, this work explicitly deals with the issue of projections, and suggests a method to deal with non-trivial projections. We show how this algorithm can be used to quantify uncertainty in discrete problems by utilising the (convex) Lovasz Extension for the min s,t-cut graph problem.