Robust Paths: Geometry and Computation

TL;DR

This paper introduces a geometric approach to efficiently analyze the trade-offs in robust optimization by characterizing a robust path as a Bregman projection, enabling approximation via standard optimization algorithms.

Contribution

It presents a novel geometric framework for robust paths in convex robust optimization, linking them to Bregman projections and standard algorithms, reducing computational effort.

Findings

Robust paths can be approximated by trajectories of standard optimization algorithms.

The approximation error depends on the geometry of the feasible region and uncertainty set.

Special cases yield zero approximation error, such as polyhedral monotone feasible regions with ellipsoidal uncertainty.

Abstract

Applying robust optimization often requires selecting an appropriate uncertainty set both in shape and size, a choice that directly affects the trade-off between average-case and worst-case performances. In practice, this calibration is usually done via trial-and-error: solving the robust optimization problem many times with different uncertainty set shapes and sizes, and examining their performance trade-off. This process is computationally expensive and ad hoc. In this work, we take a principled approach to study this issue for robust optimization problems with linear objective functions, convex feasible regions, and convex uncertainty sets. We introduce and study what we define as the robust path: a set of robust solutions obtained by varying the uncertainty set's parameters. Our central geometric insight is that a robust path can be characterized as a Bregman projection of a curve (whose geometry is defined by the uncertainty set) onto the feasible region. This leads to a surprising discovery that the robust path can be approximated via the trajectories of standard optimization algorithms, such as the proximal point method, of the deterministic counterpart problem. We give a sharp approximation error bound and show it depends on the geometry of the feasible region and the uncertainty set. We also illustrate two special cases where the approximation error is zero: the feasible region is polyhedrally monotone (e.g., a simplex feasible region under an ellipsoidal uncertainty set), or the feasible region and the uncertainty set follow a dual relationship. We demonstrate the practical impact of this approach in two settings: portfolio optimization and adversarial deep learning.

Tables

Table 1. Table 1: Computation Cost: Algorithm 2 v.s. FGSM

Method	Time per robust model (min)	Time to solve $N=100$ robust models (hrs)
Our Method: Algorithm 2	$0.25$ ($T_{\mathrm{ppm}}$)	$0.64$ ($T_{\mathrm{at}}+n{T}_{\mathrm{ppm}}$)
FGSM (Wong et al. 2020)	$15.12$ ($T_{\mathrm{at}}$)	$25.2$ ($n{T}_{\mathrm{at}}$)

Equations

$$(\mathrm{RC})\underset{x\in \mathcal{X}}{\min }\underset{a\in \mathcal{U}}{\max }⟨a,x⟩,$$

$$\mathcal{U}=\{a_0+\xi :\xi \in \Xi \subset {\mathbb{R}}^n\}.$$

$$\Xi (r,\mathcal{V})=\left\{\xi \in {\mathbb{R}}^n:\Vert \xi {\Vert }_{\mathcal{V}}\le r\right\},$$

$$(\mathrm{P})\underset{x\in \mathcal{X}}{\min }⟨a_0,x⟩.$$

$$\mathcal{P}(\mathcal{V})=\left\{x_{\mathrm{R}}(r,\mathcal{V})\in \underset{x\in \mathcal{X}}{\mathrm{argmin}}\underset{\xi \in \Xi (r,\mathcal{V})}{\max }⟨a_0+\xi ,x⟩:r\in [0,\infty )\right\}.$$

$$D_{\psi }(x,y):=\psi (x)-\psi (y)-⟨\nabla \psi (y),x-y⟩.$$

$$⟨\nabla \psi (x)-\nabla \psi (x^{\prime }),y-x^{\prime }⟩\le 0,\forall y\in \mathcal{C}.$$

$${\Pi }_{\mathcal{C}}^{\psi }={\Pi }_{\mathcal{C}}^{\psi }\circ {\Pi }_{\mathrm{Aff}(\mathcal{C})}^{\psi }$$

$$\nabla \psi \left({\Pi }_{\mathcal{K}+x_0}^{\psi }\left(y_0\right)\right)={\Pi }_{{\mathcal{K}}^{\ast }+\nabla \psi (y_0)}^{{\psi }^{\ast }}\left(\nabla \psi (x_0)\right).$$

$$x_{k+1}=\underset{x\in \mathcal{X}}{\mathrm{argmin}}⟨a_0,x⟩+{\lambda }_k{D}_{\psi }(x,x_k),k=0,1,\dots$$

$$x_{\mathrm{CP}}(\omega )=\underset{x\in \mathcal{X}}{\mathrm{argmin}}⟨a_0,x⟩+\omega D_{\psi }(x,x_0).$$

$$x_{\mathrm{R}}^{\prime }(\omega ):=x_{\mathrm{R}}^{\prime }(\omega ,\mathcal{V})=\underset{x\in \mathcal{X}}{\mathrm{argmin}}⟨a_0,x⟩+\omega \phi (x)=\underset{x\in \mathcal{X}}{\mathrm{argmin}}⟨a_0,x⟩+\omega \cdot g\circ \Vert x{\Vert }_{{\mathcal{V}}^{\circ }}.$$

$$x_{\mathrm{R}}=\underset{\omega \to \infty }{\lim }{x}_{\mathrm{R}}^{\prime }(\omega ,\mathcal{V})=\underset{x\in \mathcal{X}}{\mathrm{argmin}}\phi (x)\text{and}{x}_{\mathrm{E}}=\underset{\omega \to 0}{\lim }{x}_{\mathrm{R}}^{\prime }(\omega ,\mathcal{V})=\underset{x\in \mathcal{X}}{\mathrm{argmin}}⟨a_0,x⟩.$$

$$(\mathrm{RCD})\underset{x\in \mathcal{X}}{\min }⟨a_0,x⟩+r\Vert x{\Vert }_{{\mathcal{V}}^{\circ }}.$$

$$x_{\mathrm{R}}^{\prime }(\omega ,\mathcal{V})\in \underset{x\in \mathcal{X}}{\mathrm{argmin}}\underset{\xi \in \Xi (r(\omega ),\mathcal{V})}{\max }⟨a_0+\xi ,x⟩$$

$${\mathrm{Prox}}_{c,\mathcal{S}}^{\psi }(y,\eta )={\Pi }_{\mathcal{S}}^{\psi }\left(\nabla {\psi }^{\ast }(\nabla \psi (y)-\eta c)\right).$$

$${\mathcal{P}}^{\prime }(\mathcal{V})=\left\{{\Pi }_{\mathcal{X}}^{\phi }\left(\nabla {\phi }^{\ast }\left(\nabla \phi (0)-{\omega }^{-1}{a}_0\right)\right):\omega \in [0,\infty )\right\}$$

$$x_{\mathrm{R}}^{\prime }(\omega ,\mathcal{V})=$$

$$=$$

$$\left\{x_{\mathrm{CP}}(\omega ):\omega \in [0,\infty )\right\}=\left\{{\Pi }_{\mathcal{X}}^{\phi }\left(\nabla {\phi }^{\ast }\left(\nabla \phi (x_{\mathrm{R}})-{\omega }^{-1}{a}_0\right)\right):\omega \in [0,\infty )\right\}.$$

$$x_{k+1}={\Pi }_{\mathcal{X}}^{\phi }\left(\nabla {\phi }^{\ast }\left(\nabla \phi (x_k)-{\lambda }_k^{-1}{a}_0\right)\right),\text{for }k=0,1,\cdots .$$

$$x_{\mathrm{CP}}(\omega )=$$

$$=$$

$$x_{k+1}=$$

$$=$$

$$D_{\phi }\left({\Pi }_{\mathcal{S}}^{\phi }(x),{\Pi }_{\mathcal{S}}^{\phi }(y)\right)\le \kappa \cdot D_{\phi }\left(x,y\right),\forall x,y\in \mathrm{int}(\mathrm{dom}(\phi )).$$

$$D_{\phi }\left({\Pi }_{\mathcal{S}+d}^{\phi }(x),{\Pi }_{\mathcal{S}}^{\phi }(x)\right)\le \kappa \cdot D_{\phi }\left(x,y\right),\forall x,y\in \mathrm{int}(\mathrm{dom}(\phi )).$$

$$D_{\phi }\left(x_{\mathrm{CP}}(\omega ),x_{\mathrm{R}}^{\prime }(\omega )\right)$$

$$\text{and }{D}_{\phi }\left(x_{\mathrm{R}}^{\prime }(\omega ),x_{\mathrm{CP}}(\omega )\right)$$

$$x_{\mathrm{R}}^{\prime }(\omega )={\Pi }_{\mathcal{X}}^{\phi }\circ {\Pi }_{\mathrm{Aff}(\mathcal{X})}^{\phi }\left(\nabla {\phi }^{\ast }\left(\nabla \phi (0)-{\omega }^{-1}{a}_0\right)\right),$$

Full text

\OneAndAHalfSpacedXII\EquationsNumberedThrough\TheoremsNumberedThrough\ECRepeatTheorems\MANUSCRIPTNO

\RUNTITLE

\TITLE

Robust Paths: Geometry and Computation

\ARTICLEAUTHORS\AUTHOR

Hao Hao \AFFCarnegie Mellon University, \EMAILhaohao@andrew.cmu.edu \AUTHORPeter Zhang \AFFCarnegie Mellon University, \EMAILpyzhang@cmu.edu

\ABSTRACT

Applying robust optimization often requires selecting an appropriate uncertainty set—both in shape and size—a choice that directly affects the trade-off between average-case and worst-case performances. In practice, this calibration is usually done via trial-and-error: solving the robust optimization problem many times with different uncertainty set shapes and sizes, and examining their performance trade-off. This process is computationally expensive and ad hoc. In this work, we take a principled approach to study this issue for robust optimization problems with linear objective functions, convex feasible regions, and convex uncertainty sets. We introduce and study what we define as the robust path: a set of robust solutions obtained by varying the uncertainty set’s parameters. Our central geometric insight is that a robust path can be characterized as a Bregman projection of a curve (whose geometry is defined by the uncertainty set) onto the feasible region. This leads to a surprising discovery that the robust path can be approximated via the trajectories of standard optimization algorithms, such as the proximal point method, of the deterministic counterpart problem. We give a sharp approximation error bound and show it depends on the geometry of the feasible region and the uncertainty set. We also illustrate two special cases where the approximation error is zero: the feasible region is polyhedrally monotone (e.g., a simplex feasible region under an ellipsoidal uncertainty set), or the feasible region and the uncertainty set follow a dual relationship. We demonstrate the practical impact of this approach in two settings: portfolio optimization and adversarial deep learning. The former numerically validates the zero approximation error under favorable conditions (feasible region is polyhedrally monotone); and when the technical conditions are violated, still retains a very small error. The latter case severely breaks the linear objective condition in our theory. But our solution technology still shows strong performance: $85\%$ reduction in computational time and near-Pareto efficiency in terms of average-case and worst-case performances.

\KEYWORDS

robust optimization, proximal methods, regularization paths, Pareto efficient robust solutions

1 Introduction

We study robust optimization problems with uncertain objective functions:

[TABLE]

where decision $x$ belongs to a closed, convex and nonempty feasible region $\mathcal{X}\subseteq\mathbb{R}^{n}$, $a$ is a vector of uncertain parameters and is only known to reside in an uncertainty set:

[TABLE]

Here $a_{0}$ is the nominal vector, $\xi$ is the uncertain perturbation assumed to be in a compact and convex set $\Xi$. When deploying $(\mathrm{RC})$ in the real world, the design of the uncertainty set $\Xi$ is crucial. To this end, we consider general uncertainty sets representable via gauge function (Freund 1987, Friedlander et al. 2014, Wei et al. 2025) constraints:

[TABLE]

where the gauge function is defined as $\|v\|_{\mathcal{V}}=\inf\{t\geq 0:v\in t\mathcal{V}\}$, and the size and shape of $\Xi(r,\mathcal{V})$ can be flexibly adjusted via radius $r$ and gauge set $\mathcal{V}$ respectively. We assume $r\geq 0$ and $\mathcal{V}$ is a compact and convex set with $0\in\mathrm{int}(\mathcal{V})$. For instance, selecting $\mathcal{V}$ to be an ellipsoid or unit $l_{p}$ norm balls recovers ellipsoidal or $l_{p}$ norm type uncertainty sets.

The calibration of the uncertainty set size and shape $(r,\mathcal{V})$ directly affects the performance of the resulting robust solution, yet it is a difficult task in practice. The design of $(r,\mathcal{V})$ controls the robustness and efficiency trade-off of the deployed robust solution. At one end of the spectrum, if one assumes $r=0$, i.e., $a$ takes its nominal value $a_{0}$ deterministically, $(\mathrm{RC})$ is reduced to the following deterministic optimization problem:

[TABLE]

A resulting optimal solution $x_{\mathrm{E}}$ (E for efficiency) performs well if indeed $a=a_{0}$; however, there is no performance guarantee when the true realization of $a$ deviates from $a_{0}$. At the other end of the spectrum, a highly risk-averse decision maker may select a large $r=\overline{r}$, and solve $(\mathrm{RC})$ for a robust solution, which is robustly optimal under $a\in\mathcal{U}(\overline{r},\mathcal{V})$, yet it may perform poorly under $a=a_{0}$. The uncertainty set shape $\mathcal{V}$ is another important design lever. $\mathcal{V}$ is typically designed to leverage the available information on the uncertainty while ensuring the computational tractability of its resulting (RC).

Existing approaches calibrate $(r,\mathcal{V})$ based on probabilistic guarantees (Bertsimas and Sim 2004, Bertsimas et al. 2021, Mohajerin Esfahani and Kuhn 2018, Blanchet et al. 2019). However, such approaches assume prior knowledge of the uncertainty distribution or observations on the uncertainty, which can be unavailable in practice. Even with distributional information or data on the uncertainty, it has been observed that the resulting robust solutions from this approach can be too conservative (Sim et al. 2021). Often, the practical approach is costly and ad hoc: solving the robust optimization problem multiple times, each under a different choice of $(r,\mathcal{V})$, before comparing the performance of the different robust solutions (which includes cross-validation among other statistical methods for hyperparameter tuning) (Ben-Tal et al. 2013, Sim et al. 2021, Mohajerin Esfahani and Kuhn 2018). To this end, ideally, decision makers need the entire set of robust solutions under multiple shapes $\mathcal{V}$ and radii $r$.

Definition 1.1** **(Robust Path)

The robust path of $(\mathrm{RC})$ under $\mathcal{V}$ is defined as

[TABLE]

The challenge lies in obtaining the robust paths, potentially without repeatedly solving the robust counterpart. For any hope in tackling this problem, one first needs to study the structure of the robust paths under different $\mathcal{V}$, before exploiting the structural information to find an algorithm capable of tracing the robust paths. To this end, we investigate the following questions: 1. What are the structures of robust paths? 2. Given the structural information, can we find algorithms to trace the robust paths $\mathcal{P}(\mathcal{V})$ under different $\mathcal{V}$?

Contribution. In this paper, we answer both questions positively:

We characterize the geometry of the robust paths as the Bregman projection of curves (whose geometries are defined by $\mathcal{V}$) onto the feasible region.

• 2.

Once the appropriate geometric lens is established, we find a surprisingly simple way to approximate robust paths. We connect the following two conceptually distinct, yet geometrically similar solution sets: a) the robust paths of robust optimization problems (RC) and b) the optimization paths of the deterministic optimization counterparts (P). Specifically: 2.1. The proximal point method (PPM) optimization paths for solving the deterministic counterpart (P) initialized at the “most robust” solution are close approximate, sometimes even exact, robust paths. 2.2. The design of the robust path uncertainty set shape, $\mathcal{V}$ is equivalent to the choice of the PPM distance-generating function; adjusting the cadence of the robust solutions’ radii, $r$, corresponds exactly to adjusting the step-size of the PPM. 2.3. The distance between the PPM approximation and the exact robust path hinges precisely on the geometry of the feasible region $\mathcal{X}$ and the uncertainty set shape $\mathcal{V}$.

• 3.

We computationally approximate robust paths in two settings—portfolio optimization and adversarial deep learning—to show the quality and efficiency of the resulting computational strategies.

1.1 Related Works

Calibrating the uncertainty set design $(r,\mathcal{V})$ to balance robustness and efficiency is a major challenge in applying robust optimization to real-world problems. Bertsimas and Sim (2004) introduced the budgeted uncertainty set where the robustness and efficiency trade-off can be adjusted via a single perturbation budget hyperparameter. Under the assumption that the uncertainty follows a symmetric and bounded distribution, they provide probabilistic bounds on constraint violation as a theoretical remedy for setting the budget hyperparameter. A more recent line of work on data-driven distributionally robust optimization (Gao and Kleywegt 2023, Mohajerin Esfahani and Kuhn 2018, Blanchet et al. 2019) explicitly models the ambiguity in the uncertainty distribution and optimizes the worst-case objective over an ambiguity set of distributions within a fixed distance of the empirical distribution, e.g., a Wasserstein ball of radius $r$ centered at the empirical distribution. Mohajerin Esfahani and Kuhn (2018) provides theoretical guarantees that for a sufficiently large ambiguity set size, the ambiguity set contains the true distribution with high probability. However, the uncertainty/ambiguity set size and shape designed via probabilistic guarantees are empirically observed to be too conservative. Furthermore, in practice, many problems lack such statistical information to begin with for meaningful probabilistic guarantees. Thus, practitioners often resort to a computationally costly approach of computing multiple robust solutions under varying $(r,\mathcal{V})$, before selecting the robust solution with the best out-of-sample performance (Ben-Tal et al. 2013, Sim et al. 2021, Long et al. 2023, Mohajerin Esfahani and Kuhn 2018). We quote Ben-Tal et al. (2013): “However, it is a priori not possible to judge which uncertainty set is the ‘best’. We advocate the pragmatic approach to perform the robust optimization for different choices of uncertainty set, and then select the one that leads to the best optimal objective value.”. Therefore, we argue that the challenge is in efficiently obtaining the entire robust path. In this work, we show that the robust path of (RC) can be approximated via a single proximal path of (P), without the need to solve the (RC) multiple times. We provide approximation error bounds that depend solely on the geometry of the uncertainty set and the feasible region.

Within the robust optimization literature, the properties of the robust paths have been studied in terms of their stability and continuity. Chan and Mar (2017) and Chuong and Thanh Tinh (2025) study the stability of the robust optimization optimal value and the robust optimal solution set with regard to variations of the uncertainty set. Our work has a different goal, which is the geometrical characterization of the entire robust path and subsequently the algorithmic approximation of the robust paths of $(\mathrm{RC})$ via optimization paths of $(\mathrm{P})$. Iancu and Trichakis (2014) studies another important solution set for robust optimization. They show that under a fixed uncertainty set design $(r,\mathcal{V})$, the set of robustly (worst-case) optimal solutions is not necessarily unique; subsequently, they introduce Pareto robustly optimal solutions within the set of robustly optimal solutions: if no other solutions perform at least as well for all uncertainty realizations and strictly better for some uncertainty realizations. Our robust path is a different robust solution set under varying $r$.

Robust optimization, under some conditions, has been shown to be equivalent to regularized optimization (Xu et al. 2009, Shafieezadeh Abadeh et al. 2015, Mohajerin Esfahani and Kuhn 2018). For this reason, the robust path is closely connected to the concept of regularization path in the statistics and machine learning literatures. A regularization path is defined as the set of optimal model weights under varying regularization hyperparameters. Existing works on regularization paths can be categorized as descriptive (analyzing the structural properties and the complexity of the regularization path) (Gärtner et al. 2012, Gebken et al. 2023, Mairal and Yu 2012) and constructive (algorithms generating an approximate or exact regularization path) (Friedman et al. 2010, Hastie et al. 2004, Bach et al. 2004, Park and Hastie 2007, Ji et al. 2020). The connection between the first-order method trajectory and the regularization path (often termed explicit regularization of first-order methods) has been observed empirically (Barrett and Dherin 2021) and studied theoretically (Suggala et al. 2018). The regularization path literature focuses on unconstrained regularized optimization problems encountered in learning. Our work differs from the literature and addresses constrained robust/regularized problems. Most interestingly, under the constrained setting, we show that 1. the geometry of the robust/regularization path and 2. the quality of the approximation of robust/regularization paths of $(\mathrm{RC})$ by the optimization paths for $(\mathrm{P})$ both depend critically on the geometry of the feasible region $\mathcal{X}$. Our constrained results subsume the unconstrained setting in the case of linear objective functions.

1.2 Structure of Paper

The presentation of our results is structured as follows. Section 2 introduces key technical concepts fundamental to our analysis. In Section 3, we present a geometric lens that unifies the characterization of the robust paths of $(\mathrm{RC})$ and two optimization paths for $(\mathrm{P})$, all as Bregman projections of curves onto the feasible region. In Section 4, we show the robust paths of $(\mathrm{RC})$ can be algorithmically approximated via the optimization paths for solving $(\mathrm{P})$. The structure of the main results is summarized in Figure 1. We derive a sharp approximation error bound that depends on the geometry of the feasible region and the uncertainty set. We also present two special cases where the approximation error is zero based on polyhedral analysis and duality. In Section 5, we validate our theory in two settings: portfolio optimization and adversarial deep learning.

2 Preliminary

In this section, we review concepts that are fundamental to our analysis in Sections 3 and 4.

2.1 Notations

We adopt the notations of Rockafellar (1970). We denote the extended real numbers as $\overline{\mathbb{R}}=\mathbb{R}\cup\{-\infty,+\infty\}$, the nonnegative real numbers as $\mathbb{R}_{+}=[0,+\infty]$. Given an extended real-valued function $f:\mathbb{R}^{n}\rightarrow\overline{\mathbb{R}}$, its effective domain is defined as $\mathrm{dom}(f)=\{x\in\mathbb{R}^{n}:f(x)<+\infty\}$, its epigraph is defined as $\mathrm{epi}(f)=\{(x,t)\in\mathbb{R}^{n}\times\mathbb{R}:f(x)\leq t\}$. $f$ is proper if $f(x)>-\infty$ for all $x\in\mathbb{R}^{n}$ and there exists $x\in\mathbb{R}^{n}$ such that $f(x)<+\infty$, i.e., $\mathrm{dom}(f)\neq\varnothing$. In addition, $f$ is closed if $\mathrm{epi}(f)$ is closed. Let $\mathcal{C}\subseteq\mathbb{R}^{n}$ be a nonempty closed convex set, we denote the interior of $\mathcal{C}$ as $\mathrm{int}(\mathcal{C})$, the relative interior of $\mathcal{C}$ as $\mathrm{ri}(\mathcal{C})$, the boundary of $\mathcal{C}$ as $\mathrm{bd}(\mathcal{C})$ and the affine hull of $\mathcal{C}$ as $\mathrm{Aff}(\mathcal{C})$. The polar set of $\mathcal{C}$ is defined as $\mathcal{C}^{\circ}=\{y\in\mathbb{R}^{n}:\langle x,y\rangle\leq 1,\forall x\in\mathcal{C}\}$. In addition, we define the support function of $\mathcal{C}$ as $\sigma(\ \cdot\ |\mathcal{C})=\sup_{y\in\mathcal{C}}\langle\ \cdot\ ,y\rangle$. Let $\mathcal{V}$ be a closed and convex set containing the origin, we define the gauge function induced by $\mathcal{V}$ as $\|v\|_{\mathcal{V}}=\inf\{t\geq 0:v\in t\mathcal{V}\}$. The gauge function has the well-known dual formulation $\|v\|_{\mathcal{V}}=\sigma(v|\mathcal{V}^{\circ})$ (Rockafellar 1970, Theorem 14.5).

2.2 Smoothness and Strict Convexity of Convex Sets

Definition 2.1** **(Strict convexity and Smoothness of Convex Sets)

A convex set $\mathcal{C}$ with $\mathrm{int}(\mathcal{C})\neq\varnothing$ is strictly convex if for any distinct $x$ and $y\in\mathrm{bd}(\mathcal{C})$, $\{\alpha x+(1-\alpha)y:\alpha\in(0,1)\}\subset\mathrm{int}(\mathcal{C})$. A point $x\in\mathrm{bd}(\mathcal{C})$ is regular if the supporting hyperplane of $\mathcal{C}$ at $x$ is unique, $\mathcal{C}$ is smooth if $x$ is regular for all $x\in\mathrm{bd}(\mathcal{C})$.

2.3 Legendre Functions

We mostly work with Legendre functions as defined in Rockafellar (1970), Section 26.

Definition 2.2** **(Legendre)

Suppose $\psi:\mathbb{R}^{n}\rightarrow\mathbb{R}\cup\{+\infty\}$ is proper, closed and convex. The function $\psi$ is Legendre (or a Legendre function), if $\psi$ is both essentially smooth and essentially strictly convex, i.e., $\psi$ satisfies

• i)

$\mathrm{int}(\mathrm{dom}(\psi))\neq\varnothing$;

• ii)

$\psi$* is differentiable on $\mathrm{int}(\mathrm{dom}(\psi))$;*

• iii)

$\|\nabla\psi(x_{n})\|_{2}\rightarrow+\infty$, for any sequence $(x_{n})\subset\mathrm{int}(\mathrm{dom}(\psi))$ such that $x_{n}\rightarrow x\in\mathrm{bd}(\mathrm{dom}(\psi))$;

• iv)

$\psi$* is strictly convex on $\mathrm{int}(\mathrm{dom}(\psi))$.*

In particular, the following property of Legendre functions is useful in studying the robust path and optimization paths in both the primal and dual space as defined by a bijection $\nabla\psi$:

Lemma 2.3** **(Rockafellar (1970), Theorem 26.5)

Let a proper closed convex function $\psi:\mathbb{R}^{n}\rightarrow\mathbb{R}\cup\{+\infty\}$ be a Legendre function. Then $\nabla\psi:\mathrm{int}(\mathrm{dom}(\psi))\rightarrow\mathrm{int}(\mathrm{dom}(\psi^{*}))$ is a bijection, with $(\nabla\psi)^{-1}=\nabla\psi^{*}:\mathrm{int}(\mathrm{dom}(\psi^{*}))\rightarrow\mathrm{int}(\mathrm{dom}(\psi))$.

2.4 Bregman Divergence, Bregman Projection and Generalized Proximal Operator

Bregman divergence provides a generalized notion of proximity between two points.

Definition 2.4** **(Bregman Divergence)

Let the distance-generating function (d.g.f.), $\psi:\mathbb{R}^{n}\rightarrow\mathbb{R}\cup\{+\infty\}$ be proper closed convex and differentiable on $\mathrm{int}(\mathrm{dom}(\psi))$, the Bregman divergence induced by $\psi$, $D_{\psi}(\cdot,\cdot):\mathbb{R}^{n}\times\mathrm{int}(\mathrm{dom}(\psi))\rightarrow\mathbb{R}_{+}$ is defined as:

[TABLE]

Bregman projection generalizes the Euclidean projection.

Definition 2.5** **(Bregman Projection)

Fix a function $\psi:\mathbb{R}^{n}\rightarrow\mathbb{R}\cup\{+\infty\}$ that is proper closed convex and differentiable on $\mathrm{int}(\mathrm{dom}(\psi))$, and a closed convex set $\mathcal{S}\subset\mathrm{int}(\mathrm{dom}(\psi))$. We define the Bregman projection associated with $\psi$ of a point $y\in\mathrm{int}(\mathrm{dom}(\psi))$ onto $\mathcal{S}$ as: $\Pi^{\psi}_{\mathcal{S}}(y)=\argmin_{x\in\mathcal{S}}D_{\psi}\left(x,y\right).$

A Legendre $\psi$ ensures the existence and the uniqueness of the Bregman projection.

Lemma 2.6** **(Bauschke et al. (1997), Theorem 3.12)

If $\psi$ is Legendre, then $\Pi^{\psi}_{\mathcal{S}}(y)$ is a singleton.

Bregman projection has the following variational characterization.

Lemma 2.7** **(Bauschke et al. (1997), Proposition 3.16)

A point $x^{\prime}\in\mathcal{C}$ is $\Pi^{\psi}_{\mathcal{C}}(x)$ if and only if

[TABLE]

The Bregman projection satisfies the following property.

Lemma 2.8

Let $\psi$ be a Legendre function, let $\mathcal{C}$ be a closed convex set, then the associated Bregman projection operator satisfies

[TABLE]

Proof 2.9

Proof. We defer the proof to Appendix 7.

The Bregman projection onto a closed convex cone has the following dual characterization.

Lemma 2.10** **(Bauschke (2003), Theorem 3.1)

Let $\psi$ be a Legendre function, let $\mathcal{K}$ be a closed convex cone in $\mathrm{dom}(\psi)$ with its dual cone denoted as $\mathcal{K}^{*}$, then

[TABLE]

The Bregman proximal operator generalizes the standard proximal operator by replacing its Euclidean distance with Bregman divergence.

Definition 2.11** **(Bregman Proximal Operator)

Fix a function $\psi:\mathbb{R}^{n}\rightarrow\mathbb{R}\cup\{+\infty\}$ that is proper closed convex and differentiable on $\mathrm{int}(\mathrm{dom}(\psi))$. Let $\eta>0$. We define the Bregman proximal operator of a linear function $f(\cdot)=\langle c,\cdot\rangle$ and a closed convex set $\mathcal{S}\subset\mathrm{int}(\mathrm{dom}(\psi))$ induced by $\psi$ as: $\mathrm{Prox}^{\psi}_{c,\mathcal{S}}(y,\eta)=\argmin_{x\in\mathcal{S}}\langle c,x\rangle+\frac{1}{\eta}D_{\psi}(x,y).$

3 Unified Geometric View of Robust Path and Two Optimization Paths

In this section, we develop a unified Bregman projection view that allows us to geometrically relate the robust path of the robust counterpart $(\mathrm{RC})$, and what we will define as optimization paths for solving the deterministic counterpart $(\mathrm{P})$.

3.1 Definitions of Two Optimization Paths

We introduce two optimization paths for the deterministic optimization problem $(\mathrm{P})$ as follows. In Definition 3.1, we utilize the proximal point method, which is a fundamental tool in the analysis of modern optimization algorithms.

Definition 3.1** **(Bregman Proximal Point Method and Bregman Proximal Point Path)

Given a distance-generating function $\psi$, a step-size sequence $\{\lambda_{k}>0\}$, and a starting point $x_{0}\in\mathcal{X}$, the Bregman proximal point method for solving problem $(\mathrm{P})$ generates a sequence $\{x_{k}\}$

[TABLE]

The Bregman proximal point path is defined as the sequence $\{x_{k}\}$.

Intuitively, $x_{k+1}$ is a point in $\mathcal{X}$ that tries to minimize $\langle a_{0},x\rangle$ without deviating too much from anchor point $x_{k}$, i.e., the previous iterate. The next definition describes points that try to minimize $\langle a_{0},x\rangle$ without deviating too much from some fixed anchor point $x_{0}$.

Definition 3.2** **(Bregman Central Path)

The Bregman central path for solving problem $(\mathrm{P})$, induced by distance-generating function $\psi$ and initialized at $x_{0}$ is the set $\{x_{\mathrm{{CP}}}(\omega):\omega\in[0,\infty)\}$ defined as

[TABLE]

Throughout this paper, we work with strictly convex $D_{\psi}$ (Assumption 3.2 below). Therefore the optimal solutions are unique and we use “$=$” instead of “$\in$” in Definitions 3.1 and 3.2.

For ease of notation, we henceforth refer to the Bregman proximal point path as the proximal path (PP); and the Bregman central path as the central path (CP). We abuse the notion of central path in this paper: we assume $D_{\psi}(\cdot,x_{0})$ is not necessarily a self-concordant barrier function of the feasible region $\mathcal{X}$, hence the central path is not necessarily strictly contained in the interior of the feasible region $\mathcal{X}$.

3.2 The Geometric View

We make the following assumptions throughout the remainder of the paper. {assumption}

The uncertainty set $\mathcal{V}$ is compact, smooth, and strictly convex, with $0\in\mathrm{int}(\mathcal{V})$.

{assumption}

Define $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$, where $g:\mathbb{R}\rightarrow\mathbb{R}_{+}$ is Legendre, $g(0)=0$ and $\nabla g(0)=0$.

Note that while Assumption 3.2 places a requirement on the uncertainty set of $(\mathrm{RC})$, Assumption 3.2 is a technical assumption to facilitate proofs, not a restriction on $(\mathrm{RC})$.

In Definition 1.1, we defined robust path as the set of (potentially non-unique) optimal solutions of $(\mathrm{RC})$ with different radii. Now with the help of Assumptions 3.2 and 3.2, we refine Definition 3.2 into Definition 3.3, where each radius $r$ corresponds to a unique optimal solution. The main implication of the solution uniqueness of $\mathcal{P}^{\prime}(\mathcal{V})$ is that its geometry can be precisely characterized via Bregman projection.

Definition 3.3** **((Characterizable) Robust Path)

The (characterizable) robust path of $(\mathrm{RC})$ under $\mathcal{V}$ is defined as $\mathcal{P}^{\prime}(\mathcal{V})=\left\{x_{\mathrm{R}}^{\prime}(\omega,\mathcal{V}):\omega\in[0,\infty)\right\},$ where

[TABLE]

The robust solution $x_{\mathrm{R}}$ and the efficient solution $x_{\mathrm{E}}$ are defined as

[TABLE]

Note that $x_{\mathrm{R}}^{\prime}(\omega,\mathcal{V})$ is defined as the solution of a regularized problem, not directly of a robust optimization problem. This is possible due to the following duality results.

Lemma 3.4

The dual problem of $(\mathrm{RC})$ under $\Xi(r,\mathcal{V})=\left\{\xi\in\mathbb{R}^{n}:\|\xi\|_{\mathcal{V}}\leq r\right\}$ is

[TABLE]

In addition, strong duality holds under $0\in\mathrm{int}(\mathcal{V})$.

Lemma 3.5

The following statements are true:

• (i)

The path in Definition 3.3 is indeed a set of robust solutions: $\mathcal{P}^{\prime}(\mathcal{V})\subset\mathcal{P}(\mathcal{V})$.

• (ii)

Given any solution $x_{\mathrm{R}}^{\prime}(\omega,\mathcal{V})$ on the path corresponding to a specific regularization strength $\omega$, the corresponding robust optimization uncertainty set radius can be identified. In other words,

[TABLE]

where $r(\omega)=\omega\nabla g(\|x_{\mathrm{R}}^{\prime}\left(\omega,\mathcal{V})\|_{\mathcal{V}^{\circ}}\right)$.

We defer the proofs of Lemmas 3.4 and 3.5 to Appendix 8.

Now we are ready for the first main result. Theorem 3.6 reveals that the robust path $\mathcal{P}^{\prime}(\mathcal{V})$ of $(\mathrm{RC})$ and two appropriately defined optimization paths of $(\mathrm{P})$ are in fact geometrically similar under the lens of Bregman projection.

Theorem 3.6

*Denote $\mathcal{P}^{\prime}(\mathcal{V})$ as the robust path of $(\mathrm{RC})$ according to Definition 3.3. Define $x_{\mathrm{R}}=\lim_{\omega\rightarrow\infty}x_{\mathrm{R}}^{\prime}(\omega,\mathcal{V})=\argmin_{x\in\mathcal{X}}\varphi(x)$. Denote $\left\{x_{\mathrm{{CP}}}(\omega):\omega\in[0,\infty)\right\}$ as the central path for solving $(\mathrm{P})$, using $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$ as the d.g.f. and initialized at $x_{\mathrm{R}}$. Denote $\left\{x_{k}\right\}$ as the proximal path for solving $(\mathrm{P})$, using $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$ as the d.g.f., under a step-size sequence $\{\lambda_{k}>0:\,\sum_{k=0}^{\infty}\lambda_{k}^{-1}=\infty\}$ and initialized at $x_{\mathrm{R}}$. The three paths have the following Bregman projection interpretation:

$\begin{aligned} \text{\emph{(Robust Path)}}\hskip 85.35826pt\mathcal{P}^{\prime}(\mathcal{V})&=\left\{\Pi^{\varphi}_{\mathcal{X}}\left(\nabla\varphi^{*}\left(\nabla\varphi(\ 0\ )-\omega^{-1}a_{0}\right)\right):\omega\in[0,\infty)\right\},\\ \text{\emph{({Central Path})}}\quad\left\{x_{\mathrm{{CP}}}(\omega):\omega\in[0,\infty)\right\}\!&=\left\{\Pi^{\varphi}_{\mathcal{X}}\left(\nabla\varphi^{*}\left(\nabla\varphi(x_{\mathrm{R}})-\omega^{-1}a_{0}\right)\right):\omega\in[0,\infty)\right\},\\ \text{\emph{({Proximal Path})}}\hskip 89.91081ptx_{k+1}\!&=\;\;\,\Pi^{\varphi}_{\mathcal{X}}\left(\nabla\varphi^{*}\left(\nabla\varphi(x_{k})-\lambda_{k}^{-1}\ a_{0}\right)\right),\;\;\text{for }k=0,1,\cdots,\ x_{0}=x_{\mathrm{R}}.\end{aligned}$

Remark. Theorem 3.6 reveals that the robust path, the central path, and the proximal path can be characterized as the Bregman projection of a curve (induced by a ray in the dual space $\nabla\varphi$) onto the feasible set $\mathcal{X}$. More specifically, the robust path and the central path only differ in their respective initial points of the curves, i.e., $\nabla\varphi(0)$ and $\nabla\varphi(x_{\mathrm{R}})$. Thus, as one may expect and as we will show in Section 4, central paths are approximate, sometimes exact robust paths. Further, the “distance” between the central path and the robust path depends on the “distance” between the robust solution $x_{\mathrm{R}}$ and the origin [math] in a precise and non-trivial manner. We also point out that the central path can be viewed as the Bregman projection of a curve onto $\mathcal{X}$, while the proximal path is generated via successive Bregman projection of small curve segments onto $\mathcal{X}$. In Section 4, we show the proximal paths are approximate, sometimes exact central paths. Finally, leveraging the central paths as intermediaries, we show that the proximal paths are approximate, under some conditions, exact robust paths.

In Figure 2(a), we visualize, as Theorem 3.6 entails, the Bregman projection interpretation of robust path $\mathcal{P}^{\prime}(\mathcal{V})$ of $\mathrm{(RC)}$ together with the two optimization paths: central path $\{x_{\mathrm{{CP}}}(\omega)\}$ and proximal path $\{x_{k}\}$ of $(\mathrm{P})$ for a single $\mathcal{V}$. The distance-generating function $\varphi$ is induced by $\mathcal{V}$ (i.e., $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$). Figure 2(b) presents the geometry of a set of robust paths $\mathcal{P}^{\prime}(\mathcal{V})$ under varying $\mathcal{V}$ designs.

3.3 Proof of Theorem 3.6

We prove the representation of robust path in Section 3.3.1, and prove the representations of central path and proximal path in Section 3.3.2.

3.3.1 Robust Path: Bregman Projection and Dual Space.

We prove the robust path’s representation in Theorem 3.6 via proving three claims. In Claim 1, we show that when the uncertainty set and the post-composition function $g$ are nice, they induce a nice (Legendre) distance-generating function $\varphi$. In Claim 2, we show that with a Legendre d.g.f., the Bregman proximal operator is a Bregman projection. In Claim 3, we show that since the robust path as defined in Definition 3.3 satisfies these properties, it can be represented as a Bregman projection of a curve induced by a ray in the dual space $\nabla\varphi$.

Claim 1

If $\mathcal{V}$ and $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$ satisfy Assumption 3.2 and Assumption 3.2 respectively, we have

• (A) Well-defined:

$\mathcal{X}\subseteq\mathrm{dom}(\varphi)$, i.e., $\varphi$ is well-defined over $\mathcal{X}$.

• (B) Legendre:

$\varphi$* is Legendre.*

• (C) Zero at Origin:

$\varphi(0)=0$* and $\nabla\varphi(0)=0$.*

Proof 3.7

Proof. (A) Well-defined. $\mathrm{dom}(\varphi)=\mathrm{dom}(\|\cdot\|_{\mathcal{V}^{\circ}})=\Pi_{\mathbb{R}^{n}}(\mathrm{epi}(\|\cdot\|_{\mathcal{V}^{\circ}}))=\Pi_{\mathbb{R}^{n}}(\mathrm{cone}(\{x\in\mathbb{R}^{n}:\|x\|_{\mathcal{V}^{\circ}}\leq 1\}\times\{1\}))=\mathbb{R}^{n}\supseteq\mathcal{X}$, where $\mathrm{cone}(\cdot)$ denotes the conic hull of a set. The second and third equalities are classic results in convex analysis, the last equality is by $0\in\mathrm{int}(\mathcal{V}^{\circ})$ (by Rockafellar (1970), Corollary 14.5.1. and the assumption that $\mathcal{V}$ is compact and convex).

(B) Legendre: Essentially strictly convex.* Consider two points $x_{1}$ and $x_{2}\in\mathrm{int}(\mathrm{dom}(\varphi))=\mathbb{R}^{n}$, denote $z=\beta x_{1}+(1-\beta)x_{2}$, for some $\beta\in(0,1)$. We show $\varphi(z)<\beta\varphi(x_{1})+(1-\beta)\varphi(x_{2})$ for two cases: $\|x_{1}\|_{\mathcal{V}^{\circ}}=\|x_{2}\|_{\mathcal{V}^{\circ}}$ and $\|x_{1}\|_{\mathcal{V}^{\circ}}\neq\|x_{2}\|_{\mathcal{V}^{\circ}}$.*

Case One $\|x_{1}\|_{\mathcal{V}^{\circ}}=\|x_{2}\|_{\mathcal{V}^{\circ}}=\alpha$: Given $\mathcal{V}$ is closed, convex and smooth with $0\in\mathcal{V}$, $\mathcal{V}^{\circ}$ is closed and strictly convex (Hiriart-Urruty and Lemaréchal 2001, Proposition 3.2.7). Consequently, the sublevel set $\mathcal{L}_{\alpha}=\{x\in\mathbb{R}^{n}:\|x\|_{\mathcal{V}^{\circ}}\leq\alpha\}=\alpha\cdot\mathcal{V}^{\circ}$ is strictly convex, hence $z\in\mathrm{int}(\mathcal{L}_{\alpha})$. In addition, given $\|\cdot\|_{\mathcal{V}^{\circ}}$ is continuous and homogeneous of degree of 1, we have $\|z\|_{\mathcal{V}^{\circ}}<\alpha$ or equivalently $\|z\|_{\mathcal{V}^{\circ}}<\beta\|x_{1}\|_{\mathcal{V}^{\circ}}+(1-\beta)\|x_{2}\|_{\mathcal{V}^{\circ}}$. Using the fact that $g$ is essentially strictly convex with $\nabla g(0)=0$, $g$ is monotonically increasing over $\mathbb{R}_{+}$. Therefore we have: $g\circ\|z\|_{\mathcal{V}^{\circ}}<g(\beta\|x_{1}\|_{\mathcal{V}^{\circ}}+(1-\beta)\|x_{2}\|_{\mathcal{V}^{\circ}})=\beta g\circ\|x_{1}\|_{\mathcal{V}^{\circ}}+(1-\beta)g\circ\|x_{2}\|_{\mathcal{V}^{\circ}}.$

Case Two $\|x_{1}\|_{\mathcal{V}^{\circ}}\neq\|x_{2}\|_{\mathcal{V}^{\circ}}$: Without loss of generality, we assume $\|x_{1}\|_{\mathcal{V}^{\circ}}<\|x_{2}\|_{\mathcal{V}^{\circ}}$. By the convexity of $\|\cdot\|_{\mathcal{V}^{\circ}}$: $\|z\|_{\mathcal{V}^{\circ}}\leq\beta\|x_{1}\|_{\mathcal{V}^{\circ}}+(1-\beta)\|x_{2}\|_{\mathcal{V}^{\circ}}.$ Given $g$ is monotonically increasing over $\mathbb{R}_{+}$ and essentially strictly convex: $g\circ\|z\|_{\mathcal{V}^{\circ}}\leq g(\beta\|x_{1}\|_{\mathcal{V}^{\circ}}+(1-\beta)\|x_{2}\|_{\mathcal{V}^{\circ}})<\beta g\circ\|x_{1}\|_{\mathcal{V}^{\circ}}+(1-\beta)g\circ\|x_{2}\|_{\mathcal{V}^{\circ}}.$

(B) Legendre: Differentiable.* Towards proving $\varphi$ is essentially smooth, we begin by showing $\varphi$ is differentiable on $\mathrm{int}(\mathrm{dom}(\varphi))=\mathbb{R}^{n}$. Since $\mathcal{V}$ is strictly convex, by invoking Schneider (2013), Corollary 1.7.3 we have $\|\cdot\|_{\mathcal{V}^{\circ}}$ is differentiable on $\mathbb{R}^{n}\setminus\{0\}$. Additionally, given $g$ is differentiable on $\mathbb{R}$, $g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$ is differentiable on $\mathbb{R}^{n}\setminus\{0\}$. It remains to show $\varphi$ is differentiable at [math], which is equivalent to verifying that the normal cone of $\mathrm{epi}(\varphi)$ at $(0,\varphi(0))$ is a single ray. Specifically, we show $\{\gamma(0,-1):\gamma\geq 0\}$ is the unique ray in $\mathcal{N}_{\mathrm{epi}(\varphi)}((0,g\circ\|0\|_{\mathcal{V}^{\circ}}))=\mathcal{N}_{\mathrm{epi}(\varphi)}((0,0))$ (i.e., $\nabla\varphi(0)=0$). We begin by showing $(0,-1)\in\mathcal{N}_{\mathrm{epi}(\varphi)}((0,0))$. For any $(x,t)\in\mathrm{epi}(\varphi)$, $\langle(0,-1),(x,t)-(0,0)\rangle=-t\leq 0$ (Given $\mathrm{epi}(\varphi)\subset\mathbb{R}^{n}\times\mathbb{R}_{+}$ ).*

Next we show that for any $(v,-1)$ such that $v\neq 0$, it holds that $(v,-1)\notin\mathcal{N}_{\mathrm{epi}(\varphi)}((0,0))$. Let $x=\alpha v$ for some sufficiently small $\alpha>0$, we have $(x,\varphi(x))=(\alpha v,g(\|\alpha v\|_{\mathcal{V}^{\circ}}))=(\alpha v,g(\alpha\|v\|_{\mathcal{V}^{\circ}}))\in\mathrm{epi}(\varphi)$. We are interested in if the following dot product is positive $\langle(v,-1),(x,\varphi(x))-(0,0)\rangle=\langle(v,-1),(x,\varphi(x))\rangle=\alpha\|v\|_{2}^{2}-g(\alpha\|v\|_{\mathcal{V}^{\circ}})$. Since $\alpha>0$, we can equivalently show $\|v\|_{2}^{2}-\frac{g(\alpha\|v\|_{\mathcal{V}^{\circ}})}{\alpha}$ is positive for some sufficiently small $\alpha$. Indeed, $\lim_{\alpha\rightarrow 0^{+}}\|v\|_{2}^{2}-\frac{g(\alpha\|v\|_{\mathcal{V}^{\circ}})}{\alpha}=\|v\|_{2}^{2}-\lim_{\alpha\rightarrow 0^{+}}\frac{g(\alpha\|v\|_{\mathcal{V}^{\circ}})}{\alpha}=\|v\|_{2}^{2}-\lim_{\gamma\rightarrow 0^{+}}\frac{g(\gamma)}{\gamma}\|v\|_{\mathcal{V}^{\circ}}=\|v\|^{2}_{2}>0,$ where the third equality is by the differentiability of $g$ at [math], specifically $\nabla g(0)=0$. The inequality is due to $v\neq 0$. In conclusion, $\varphi$ is differentiable on $\mathbb{R}^{n}$, with $\nabla\varphi(0)=0$.

(B) Legendre: Essentially smooth.* Having proved the differentiability of $\varphi$ on $\mathbb{R}^{n}$, to conclude $\varphi$ is essentially smooth, it suffice to show $\|\nabla\varphi(x_{n})\|_{2}\rightarrow+\infty$, for any sequence $(x_{n})\subset\mathrm{int}(\mathrm{dom}(\varphi))$ such that $x_{n}\rightarrow x\in\mathrm{bd}(\mathrm{dom}(\varphi))$. Given $\mathrm{dom}(\varphi)=\mathbb{R}^{n}$, $\mathrm{bd}(\mathrm{dom}(\varphi))=\varnothing$, the condition is vacuously satisfied.*

(C) Zero at Origin.* We have proved $\varphi(0)=0$ and $\nabla\varphi(0)=0$ in the course of proving the differentiability of $\varphi$. \Halmos*

Then we prove that under a Legendre distance generating function, the induced Bregman proximal operator has a Bregman projection interpretation.

Claim 2** **(Bregman Proximal Operator is Bregman Projection)

Fix a function $\psi:\mathbb{R}^{n}\rightarrow\mathbb{R}\cup\{+\infty\}$ that is Legendre, and a closed convex set $\mathcal{S}\subset\mathrm{int}(\mathrm{dom}(\psi))$. Then

[TABLE]

Proof 3.8

Proof. Denote $\overline{x}=\mathrm{Prox}^{\psi}_{c,\mathcal{S}}(y,\eta)=\argmin_{x\in\mathcal{S}}\langle c,x\rangle+\frac{1}{\eta}D_{\psi}(x,y)$. The minimizer is unique due to strong convexity of Legendre distance-generating functions (Assumption 3.2). By the first order optimality condition, $\langle\eta c+\nabla\psi(\overline{x})-\nabla\psi(y),x^{\prime}-\overline{x}\rangle\geq 0,\ \forall x^{\prime}\in\mathcal{S}$, which is exactly the first order optimality condition (Lemma 2.7) for $\overline{x}=\Pi^{\psi}_{\mathcal{S}}\left((\nabla\psi)^{-1}(\nabla\psi(y)-\eta c)\right)=\Pi^{\psi}_{\mathcal{S}}\left(\nabla\psi^{*}(\nabla\psi(y)-\eta c)\right).$ \Halmos

Claim 3

The robust path $\mathcal{P}^{\prime}(\mathcal{V})$ of $(\mathrm{RC})$ is equivalent to the Bregman Projection induced by $\varphi$ of curve $\left\{\nabla\varphi^{*}\left(\nabla\varphi(0)-\omega^{-1}a_{0}\right):\omega\in[0,\infty)\right\}$ onto the feasible region $\mathcal{X}$:

[TABLE]

Proof 3.9

Proof. This result follows directly from the previous claims.

[TABLE]

The second equality is due to Claim 1 (C). The third equality is by Definition 2.4. The fourth equality is due to Claim 1 (B) and Claim 2. \Halmos

3.3.2 Central Path and Proximal Path: Bregman Projection and Dual Space.

Claim 4

• Central Path:

The central path (CP) $\{x_{\mathrm{{CP}}}(\omega):\omega\in[0,\infty)\}$ induced by the distance-generating function $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$ initialized at $x_{\mathrm{R}}$ can be formulated as the Bregman projection of the curve $\left\{\nabla\varphi^{*}\left(\nabla\varphi(x_{\mathrm{R}})-\omega^{-1}a_{0}\right):\omega\in[0,\infty)\right\}$ onto the feasible region $\mathcal{X}$:

[TABLE]

• Proximal Path:

The proximal path (PP) $\{x_{k}\}$ of $(\mathrm{P})$ induced by the distance-generating function $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$ initialized at $x_{\mathrm{R}}$ and associated with sequence $\{\lambda_{k}\}$ can be formulated as successive Bregman projections onto the feasible region $\mathcal{X}$:

[TABLE]

Proof 3.10

Proof. The proof is a straightforward application of Definition 2.11 and Claim 2.

[TABLE]

[TABLE]

\Halmos

4 Recovering Robust Path by Optimization Paths

In this section, leveraging the geometric view of Section 3, we show that the robust path $\mathcal{P}^{\prime}(\mathcal{V})$ of $(\mathrm{RC})$ can be approximated by some appropriately designed central path $\{x_{\mathrm{{CP}}}(\omega)\}$ and proximal path $\{x_{k}\}$ of $(\mathrm{P})$. The two optimization paths’ distance-generating functions $\varphi$ are induced by the uncertainty set shape $\mathcal{V}$, i.e., $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$. Furthermore, the two optimization paths are initialized at the most robust solution $x_{\mathrm{R}}$, while solving for the most efficient solution $x_{\mathrm{E}}$.

The structure of the main results in this section is summarized in Figure 1. In Section 4.1, we establish that central paths are good approximations of, and sometimes exact, robust paths (Theorem 4.2 and Corollary 4.5). This builds on the previous insight that the robust path and the central path are both Bregman projections of similar curves onto the feasible region, where they differ only in the initial points of the curves (Theorem 3.6).

Theorem 3.6 also points out that the central path and the proximal path share the same initial point, but differ only in that the central path is the Bregman projection of an entire curve onto the feasible region, while the proximal path is generated via successive Bregman projections of short curve segments onto the same feasible region. Building on this insight, in Section 4.2, we first show two special cases where central paths are exact proximal paths (Propositions 4.9 and 4.11). We then build on this analysis to establish a general bound between central paths and proximal paths (Theorem 4.13).

In Section 4.3, we close the loop and give sufficient conditions under which proximal paths are exact robust paths (Theorem 4.15). This theorem also quantifies the exact relationship between the step-size progression on a proximal path and the uncertainty set radii on the robust path.

Finally, inspired by this theory, we state an algorithm for (approximately) recovering a robust path via a single algorithmic pass of the proximal method. Numerical validations in the subsequent Section 5 show that this algorithm verifies the predictions of our theory when the problem’s technical setup matches our theory. Surprisingly, we also observe that this algorithm retains a strong performance (in terms of solution quality and computational time) even when the technical conditions are severely violated.

4.1 Robust Path and Central Path

We define the $\kappa$-expansiveness of a Bregman projection operator as follows.

Definition 4.1

Give a distance-generating function, $\varphi$, the induced Bregman projection $\Pi^{\varphi}_{\mathcal{S}}:\mathrm{int}(\mathrm{dom}(\varphi))\rightarrow\mathcal{S}$ is $\kappa$-expansive if for any closed and convex $\mathcal{S}$, the following inequalities hold:

[TABLE]

In addition, denote $d=y-x$:

[TABLE]

For instance, for $\varphi=\frac{1}{2}\|x\|^{2}_{2}$, $\Pi^{\varphi}_{\mathcal{S}}$ is the usual Euclidean projection and is 1-expansive. More information can be found in Appendix 9.

We now show that the robust path $\mathcal{P}^{\prime}(\mathcal{V})$ of $(\mathrm{RC})$ can be approximated by the central path $\{x_{\mathrm{{CP}}}(\omega)\}$ for $(\mathrm{P})$ induced by $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$ and initialized at $x_{\mathrm{R}}$. Its proof relies on the geometric view on both $\mathcal{P}^{\prime}(\mathcal{V})$ and $\{x_{\mathrm{{CP}}}(\omega)\}$ made possible by Theorem 3.6.

Theorem 4.2

Assume $\mathcal{V}$ satisfies Assumption 3.2, $\varphi$ satisfies Assumption 3.2, and the induced Bregman projection $\Pi^{\varphi}_{\mathcal{S}}$ is $\kappa$-expansive. The Bregman divergence between the central path and the robust path is uniformly bounded by an upper bound. The upper bound depends on $\kappa$ and the Bregman divergence between two points.

[TABLE]

Moreover, the bound is sharp.

Proof 4.3

Proof. On a high-level, the proof proceeds by first mapping the objects of interest via bijection $\nabla\varphi:\mathrm{int}(\mathrm{dom}(\varphi))\rightarrow\mathrm{int}(\mathrm{dom}(\varphi^{*}))$ from the primal space to the dual space where the paths enjoy simple structure, before mapping the objects via $(\nabla\varphi)^{-1}=\nabla\varphi^{*}:\mathrm{int}(\mathrm{dom}(\varphi^{*}))\rightarrow\mathrm{int}(\mathrm{dom}(\varphi))$ back to the primal space to establish the result.

We begin by taking the geometry view of the two paths of Theorem 3.6. Together with Lemma 2.8, we have

[TABLE]

[TABLE]

We recall by Definition 3.3, $x_{\mathrm{R}}=\argmin_{x\in\mathcal{X}}\varphi(x)$; together with Claim 1.C, we have $x_{\mathrm{R}}=\Pi^{\varphi}_{\mathcal{X}}(0)$.

Define $x_{\mathrm{A}}:=\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{X})}(0)\in\mathrm{Aff}(\mathcal{X})$, we have $\mathrm{Aff}(\mathcal{X})=\mathcal{L}+x_{\mathrm{A}}$, where $\mathcal{L}=\mathrm{Aff}(\mathcal{X})-x_{\mathrm{A}}$ is a linear subspace. Now, we apply the dual perspective of Lemma 2.10 on $x_{\mathrm{A}}=\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{X})}(0)$:

[TABLE]

Hence $\nabla\varphi\left(x_{\mathrm{A}}\right)\in\mathcal{L}^{\bot}+\nabla\varphi(0)$, or equivalently $\mathcal{L}^{\bot}+\nabla\varphi(0)=\mathcal{L}^{\bot}+\nabla\varphi\left(x_{\mathrm{A}}\right)$.

Next, we map $\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{X})}\left(\nabla\varphi^{*}\left(\nabla\varphi(0)-\omega^{-1}a_{0}\right)\right)$ and $\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{X})}\left(\nabla\varphi^{*}\left(\nabla\varphi(x_{\mathrm{R}})-\omega^{-1}a_{0}\right)\right)$ of (5) and (6) to the dual space where the pair enjoy a simpler characterization of Bregman projection of $x_{\mathrm{A}}$ onto two parallel affine subspaces.

We begin by applying the dual characterization of Lemma 2.10 on $\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{X})}\left(\nabla\varphi^{*}\left(\nabla\varphi(0)-\omega^{-1}a_{0}\right)\right)$. For any $\omega\in[0,\infty)$ we have

[TABLE]

where the first equality is due to $x_{\mathrm{A}}\in\mathrm{Aff}(\mathcal{X})$, the second equality is a direct application of Lemma 2.10, the third equality is by $\mathcal{L}^{\bot}+\nabla\varphi(0)=\mathcal{L}^{\bot}+\nabla\varphi\left(x_{\mathrm{A}}\right)$.

Similarly, for $\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{X})}\left(\nabla\varphi^{*}\left(\nabla\varphi(x_{\mathrm{R}})-\omega^{-1}a_{0}\right)\right)$, we have

[TABLE]

Now we are ready to prove the Theorem:

[TABLE]

The first equality follows from (5) and (6), the first inequality is by Definition 4.1, the second equality is by Bauschke et al. (1997) Theorem 3.7(v) (mapping from the primal space to the dual space), the third equality is by (8) and (9), the second inequality is by Definition 4.1, the fourth equality is again by Bauschke et al. (1997) Theorem 3.7(v) (mapping from the dual space back to the primal space). The other inequality $D_{\varphi}\left(x_{\mathrm{R}}^{\prime}(\omega),x_{\mathrm{{CP}}}(\omega)\right)\leq\kappa^{2}\cdot D_{\varphi}\left(x_{\mathrm{A}},x_{\mathrm{R}}\right)$ follows from the same argument.

To finish the proof, we establish the sharpness of the above bound via the following example:

Example 4.4** **(Upper Bound Sharpness)

Consider following problem instance $a_{0}=(-1,1)$, $\mathcal{X}=\{(x_{1},x_{2})\in\mathbb{R}^{2}:x_{1}+2x_{2}=2,\ x_{1}\geq 0.5,\ x_{2}\geq 0\}$, with $\ell_{2}$ norm setup, $\varphi(x)=\frac{1}{2}\|x\|^{2}_{2}$. It can be easily verified that for $\omega=0.5$, $\frac{1}{2}\|x_{\mathrm{CP}}(\omega)-x_{\mathrm{R}}^{\prime}(\omega)\|^{2}_{2}=\frac{1}{2}\|\Pi_{\mathcal{X}}(0)-\Pi_{\mathrm{Aff}(\mathcal{X})}(0)\|^{2}_{2}$ or equivalently, $D_{\varphi}\left(x_{\mathrm{CP}}(\omega),x_{\mathrm{R}}^{\prime}(\omega)\right)=\kappa^{2}D_{\varphi}\left(\Pi^{\varphi}_{\mathcal{X}}(0),\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{X})}(0)\right)$, where $\kappa=1$. \Halmos

Corollary 4.5

Assume $\mathcal{V}$ satisfies Assumption 3.2, and $\varphi$ satisfies Assumption 3.2, then the following statement is true.

[TABLE]

Proof 4.6

Proof. This result follows directly from Theorem 4.2: the right hand side of Theorem 4.2’s inequality collapses to zero when the two initial points match each other, $\Pi^{\varphi}_{\mathcal{X}}(0)=\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{X})}(0)$. \Halmos

Example 4.7** **(Zero Gap Between Robust Path and Central Path)

$\Pi^{\varphi}_{\mathcal{X}}(0)=\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{X})}(0)$* for the following cases.*

• •

$\mathcal{X}$* contains 0.*

• •

$\Pi^{\varphi}$* is the Euclidean projection, and $\mathcal{X}$ is the intersection of a positively oriented affine space with the positive orthant: $\mathcal{X}=\{x:Ax=b,x\geq 0\}$ where $A^{\intercal}(AA^{\intercal})^{-1}b\geq 0$. For instance, a simplex $\mathcal{X}=\{x\in\mathbb{R}^{n}_{+}:\langle 1,x\rangle=1\}$ satisfies this criterion.*

4.2 Central Path and Proximal Path

In this section, we show the two optimization paths of $(\mathrm{P})$, i.e., the proximal path $\{x_{k}\}$ and the central path $\{x_{\mathrm{{CP}}}(\omega)\}$ are in general approximations of each other, and under two special cases equivalent.

4.2.1 Special Case One: Polyhedral Monotonicity of Feasible Regions.

For convex polyhedron feasible regions $\mathcal{X}$, we show that if the following is true: once $\{x_{k}\}$ enters a face of $\mathcal{X}$ it remains in that face, then $\{x_{k}\}=\{x_{\mathrm{{CP}}}(\omega_{k})\}$. More precisely, we adopt the following definition from González-Sanz et al. (2025).

Definition 4.8

The proximal path $\{x_{k}\}$ is monotone on convex polyhedron $\mathcal{X}$, if for any face $\mathcal{F}$ of $\mathcal{X}$: $\quad x_{k}\in\mathcal{F}\quad\Rightarrow\quad x_{k+n}\in\mathcal{F},\ \forall n\in[1,K].$

Intuitively, monotonicity can be interpreted as: once a polyhedron constraint becomes active for $x_{k}$, it stays active for all the subsequent sequences. Another equivalent interpretation is that $\{x_{k}\}$ generates a path in the partially ordered set (poset) of the faces of $\mathcal{X}$ that is nonincreasing in the set order.

Proposition 4.9

Let the feasible region $\mathcal{X}$ be a convex polyhedron. Let $\{x_{k}\}$ be a proximal path initialized by $x_{0}$ associated with the step-size sequence $\{\lambda_{k}>0:\,\sum_{k=0}^{\infty}\lambda_{k}^{-1}=\infty\}$. Let $\{x_{\mathrm{{CP}}}(\omega_{k})\}$ be the central path initialized also at $x_{0}$. If any of the following conditions is satisfied:

• (C1. Monotone):

$\{x_{k}\}$* is monotone on $\mathcal{X}$,*

• (C2. Affine Subspace):

$\mathcal{X}=\{x\in\mathbb{R}^{n}:Ax=b\}$,

• (C3. Unconstrained):

$\mathcal{X}=\mathbb{R}^{n}$,

then $x_{k+1}=x_{\mathrm{{CP}}}\left(\omega_{k+1}\right),\ \forall k\in[0,K].$ Furthermore, $\omega_{k}$ can be recovered in closed form as a function of the proximal path step-size sequence $\{\lambda_{k}\}$: $\omega_{k}=\left(\lambda_{0}^{-1}+\dots+\lambda_{k-1}^{-1}\right)^{-1}.$

Proof 4.10

Proof. We defer the proof to Appendix 10.

Remark. The monotonicity condition (C1) can be verified after generating the proximal path $\{x_{k}\}$, i.e., verify if for all inequality constraints, once a constraint is active for $x_{k}$, it remains active for $x_{k+n}$ for all $n\in[1,K]$. A canonical example that satisfies the monotonicity condition (C1) is a simplex feasible region $\mathcal{X}=\Delta^{n}=\{x\in\mathbb{R}^{n}_{+}:\langle 1,x\rangle=1\}$ under an ellipsoidal uncertainty set $\mathcal{V}$ as depicted in Figure 3(a).

4.2.2 Special Case Two: Feasible Region and Uncertainty Set are Polar Pairs.

Consider the following motivating example: if the feasible region $\mathcal{X}$ and the uncertainty set $\mathcal{V}$ are both $2$-norm balls, then $\{x_{k}\}$ is not monotone on $\mathcal{X}$ while $\{x_{k}\}\subset\{x_{\mathrm{{CP}}}\left(\omega\right):\omega\in[0,\infty)\}$. Most interestingly, this observation generalizes: assume the feasible region $\mathcal{X}$ can be defined as the sub-level set of a gauge function $\|\cdot\|_{\mathcal{U}}$, if the uncertainty set is chosen as $\mathcal{V}=\mathcal{U}^{\circ}$, *i.e.*the feasible region and the uncertainty set are polar pairs up to a rescaling, then we have $\{x_{k}\}\subset\{x_{\mathrm{{CP}}}\left(\omega\right):\omega\in[0,\infty)\}$. Such an example is shown in Figure 3(b). Formally, we have the following result.

Proposition 4.11

Assume $\mathcal{X}=\{x\in\mathbb{R}^{n}:\ \|x\|_{\mathcal{U}}\leq l\}$ where $\mathcal{U}$ satisfies Assumption 3.2, if robust path uncertainty set is designed to be $\mathcal{U}^{\circ}$ and the corresponding optimization path distance-generating function $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{U}}$ satisfies Assumption 3.2, then: $\{x_{k}\}\subset\{x_{\mathrm{{CP}}}\left(\omega\right):\omega\in[0,\infty)\}.$

Proof 4.12

Proof. We defer the proof to Appendix 11.

4.2.3 General Case.

In general, the proximal path and the central path do not coincide. To this end, we establish a theoretical characterization of the distance between the two algorithmic paths with the next result. Reaching the result relies solely on our established results of Corollary 4.5 and Proposition 4.9. Intuitively, we can partition the pair of paths by the sequence of minimal faces they traverse and analyze the distance between the pair locally on each minimal face.

Theorem 4.13

Under Assumptions 1 and 2, denote $\{x_{k}\}=\{x_{k}:k\in[0,K]\}$ as the proximal path induced by $\varphi$ whose Bregman projection is $\kappa$-expansive, initialized at $x_{0}$ and associated with a step-size sequence $\{\lambda_{k}>0:\,\sum_{k=0}^{\infty}\lambda_{k}^{-1}=\infty\}$. Denote $\upsilon=\omega^{-1}$ and let $\{x_{\mathrm{{CP}}}(\upsilon^{-1};x_{0})\}=\{x_{\mathrm{{CP}}}(\upsilon^{-1};x_{0}):\upsilon\in[0,\infty)\}$ be the central path induced by $\varphi$ and initialized at ${x_{0}}$. We assume $\{x_{k}\},\{x_{\mathrm{{CP}}}(\upsilon^{-1};x_{0})\}\subset\bigcup_{i\in[I]}\mathrm{ri}\left(\mathcal{F}_{i}\right)$, where $\mathcal{F}_{i},\ \forall i\in[I]$ are faces of $\mathcal{X}$. For every $i\in[I]$, define $\mathcal{K}^{(i)}=\left[\underline{k}^{(i)},\overline{k}^{(i)}\right]=\left\{k\in[0,K]:x_{k}\in\mathrm{ri}\left(\mathcal{F}_{i}\right)\right\}$ and $\Upsilon^{(i)}=\left[\underline{\upsilon}^{(i)},\overline{\upsilon}^{(i)}\right]=\left\{\upsilon\in[0,\infty):x_{\mathrm{{CP}}}(\upsilon^{-1};x_{0})\in\mathrm{ri}\left(\mathcal{F}_{i}\right)\right\}$, consequently $\left\{x_{k}:k\in\mathcal{K}^{(i)}\right\}_{i\in[I]}$ and $\left\{x_{\mathrm{{CP}}}(\upsilon^{-1};x_{0}):\upsilon\in\Upsilon^{(i)}\right\}_{i\in[I]}$ form partitions of $\{x_{k}\}$ and $\{x_{\mathrm{{CP}}}(\upsilon^{-1};x_{0})\}$ respectively. Then, for each $i\in[I]$ we have

[TABLE]

where $\upsilon_{k}=\underline{\upsilon}^{(i)}+\sum_{j=\underline{k}^{(i)}}^{k-1}\lambda_{j}^{-1}$.

Proof 4.14

Proof. We defer the proof to Appendix 12.

4.3 Sufficient Condition: Proximal Paths are Exact Robust Paths

In this section, we close the loop and give a sufficient condition for the proximal path to be an exact robust path.

Theorem 4.15

Let $\{x_{k}\}$ be the proximal path for $\mathrm{(P)}$ induced by $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}$, initialized at $x_{\mathrm{R}}$ and associated with $\{\lambda_{k}>0:\,\sum_{k=0}^{\infty}\lambda_{k}^{-1}=\infty\}$. Assume $\Pi^{\varphi}_{\mathcal{X}}(0)=\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{X})}(0)$ and $\{x_{k}\}$ is monotone on $\mathcal{X}$, then for every $k$, $x_{k}$ is a solution to $(\mathrm{RC})$:

[TABLE]

where the corresponding uncertainty set radius $r_{k}$ admits the following closed-form expression: $r_{k}=\omega_{k}\nabla g\left(\|x_{k}\|_{\mathcal{V}^{\circ}}\right),$ with $\omega_{k}=\left(\lambda_{0}^{-1}+\dots+\lambda_{k-1}^{-1}\right)^{-1}.$

Proof 4.16

Proof. The result is a direct consequence of Corollary 4.5 and Proposition 4.9:

[TABLE]

where $r_{k}=\omega_{k}\nabla g\left(\|x_{k}\|_{\mathcal{V}^{\circ}}\right)$ by Lemma 3.5, and $\omega_{k}=\left(\lambda_{0}^{-1}+\dots+\lambda_{k-1}^{-1}\right)^{-1}$ by Proposition 4.9. \Halmos

Remark. Theorem 4.15 gives the following algorithmic insight for generating robust paths of (RC) via proximal paths of (P): (1) the design of the robust path uncertainty set shape, $\mathcal{V}$ is equivalent to the choice of the proximal path distance-generating function, $\varphi$; (2) adjusting the cadence of the robust solutions’ radii, $r$ corresponds to adjusting the step-size of the proximal path, Theorem 4.15 provides a closed-form expression of the robust solutions’ $r$ as a function of the proximal path solutions and the step-size sequence.

4.4 Algorithm: Recovering Robust Path Approximately via Proximal Path

Theorem 4.15 directly points to Algorithm 1 for recovering an (approximate) robust path of $(\mathrm{RC})$ via a single proximal path of $(\mathrm{P})$: select an appropriate step-size sequence $\{\lambda_{k}\}$ and construct an proximal path distance generating function from the robust path uncertainty set, i.e., $\varphi(\cdot)=g\circ\|\cdot\|_{\mathcal{V}^{\circ}}:\mathcal{X}\rightarrow\mathbb{R}$ satisfying Assumption 3.2; solve for the robust solution $x_{\mathrm{R}}$; initialized at $x_{\mathrm{R}}$, generate a proximal path of ($\mathrm{P}$) associated with $\{\lambda_{k}\}$ and $\varphi$; the resulting proximal path is an (approximate) robust path of ($\mathrm{RC}$), where the corresponding uncertainty set radius is a closed-form function of the proximal path solutions and the step-size sequence.

Remark. If the conditions of Theorem 4.15 are satisfied, Algorithm 1 generates a proximal path that is an exact robust path. More generally, Algorithm 1 produces a proximal path that is an approximate robust path, where the approximation error between the two paths can be characterized via Theorem 4.2 and Theorem 4.13. We also point out that an exact proximal point step is of the same computational cost as solving a single $(\mathrm{RC})$, to lower the computational cost, the exact proximal point step of Algorithm 1 can be replaced by its computationally cheaper approximations (e.g., projected gradient descent Parikh et al. (2014)) to trade higher approximate errors for lower computational costs.

5 Numerical Validations

In the previous sections, we developed a theory for robust paths and related optimization paths; the theoretical insights are operationalized as Algorithm 1. In this section, we validate the results of our theorems and the performance of Algorithm 1 via numerical experiments.

The first experiment in Section 5.1 on portfolio optimization illustrates how Algorithm 1 generates an entire set of approximate, sometimes exact, Pareto efficient portfolios via a single proximal path. We compare Algorithm 1 generated portfolios with exact Pareto efficient portfolios under three types of setups: a hyperplane feasible region, a polyhedrally monotone feasible region, and a more general feasible region with extra practical investment constraints. The hyperplane feasible region shows as special cases of Theorem 4.15, under affine subspace feasible regions or unconstrained problems, the proximal paths are exact robust paths. The polyhedrally monotone feasible region confirms under the conditions of Theorem 4.15, the proximal paths are exact robust paths. The general feasible region setup confirms the results of Theorem 4.2 and Theorem 4.13 that proximal paths in general can be good approximate robust paths. As a side product, our results extend the classical Two-Fund Theorem in finance (Markowitz 2008), with our Algorithm 1 accommodating practical trading constraints while the original theorem in finance does not.

The second experiment in Section 5.2 explores settings that significantly deviate from our theoretical assumptions, with highly nonlinear loss functions in deep learning. Even here, our theory and algorithm (correctly initializing, and sometimes restarting standard gradient-based methods) lead to orders of magnitude computational speedup while maintaining strong worst-case and nominal prediction performances.

5.1 First Experiment: Portfolio Optimization

In portfolio optimization, we are concerned with constructing a portfolio from $n$ risky assets. The return of the $n$ assets is modeled by a random vector, $\alpha$. We assume that from historical data, we can estimate the expectation and the covariance matrix of $\alpha$ to be $\mu$ and $\Sigma$. It is known that the classical Markowitz mean-variance portfolio optimization problem can be cast in the form of our $(\mathrm{RC})$ with an ellipsoidal uncertainty set $\mathcal{U}(r)=\{\mu+\xi\in\mathbb{R}^{n}:\ \|\Sigma^{-1/2}\xi\|_{2}\leq r\}$ (Natarajan et al. 2009):

[TABLE]

5.1.1 Proximal Paths are Almost Pareto Efficient Portfolios: Visualization in Solution Space.

We compute the exact Pareto efficient portfolios (equivalently, the robust path) of problem (10). In addition, we compute two approximations of the set of Pareto efficient portfolios: (i) Two-Fund Theorem (Markowitz 2008): convex combinations of the min-variance $x_{\mathrm{R}}$ and the max-return portfolio, $x_{\mathrm{E}}$ and (ii) Algorithm 1: proximal path initialized at $x_{\mathrm{R}}$ and converging to $x_{\mathrm{E}}$. We run the above experiment under increasingly more general asset weights feasible regions, $\mathcal{X}$: hyperplane, simplex, and simplex with additional trading constraints.

Figure 4 presents the results on two small instances where a portfolio is constructed with three (top row) and four (bottom row) risky assets. The results give the following insights:

• (i)

Under general feasible regions $\mathcal{X}=\{x\in\mathbb{R}^{n}:\langle 1,x\rangle=1,x_{\mathrm{lb}}\leq x\leq x_{\mathrm{ub}}\}$: As depicted in Figure 4, (a3) and (b3), the proximal paths are higher quality approximations of Pareto efficient portfolios than convex combinations. For instance, Figure 4, (a3) shows a degenerate instance where the proximal path is not monotone on $\mathcal{X}$ (entered one edge from another edge), but remains an exact set of Pareto efficient portfolios. Figure 4, (b3) shows a general instance where the proximal path is a set of approximate Pareto efficient portfolios, where the approximation error bound can be characterized by Theorem 4.2 and Theorem 4.13.

• (ii)

Under simplex feasible regions $\mathcal{X}=\{x\in\mathbb{R}^{n}:\langle 1,x\rangle=1,x\geq 0\}$: As shown in Figure 4, (a2) and (b2), the set of Pareto efficient portfolios has a piecewise linear structure, under which the Two-Fund Theorem no longer holds and convex combinations only generate poor approximations of the Pareto efficient portfolios. In contrast, by Proposition 4.9, the proximal paths are monotone on simplices; hence, the proximal paths are exact Pareto efficient portfolios.

• (iii)

Under hyperplane feasible regions $\mathcal{X}=\{x\in\mathbb{R}^{n}:\langle 1,x\rangle=1\}$: the Two-Fund Theorem (Markowitz 2008) states that the set of Pareto efficient portfolios can be constructed exactly as convex combinations of the min-variance portfolio, $x_{\mathrm{R}}$, and the max-return portfolio, $x_{\mathrm{R}}$. Our Theorem 3.6 gives a geometric proof of the Two-Fund Theorem, as depicted in Figure 4, (a1) and (b1): under hyperplane feasible regions, the set of Pareto efficient portfolios is a line segment in $\mathbb{R}^{n}$ which can be generated as convex combinations of any two Pareto efficient portfolio including $x_{\mathrm{R}}$ and $x_{\mathrm{E}}$. In addition, by Proposition 4.9, the proximal paths are monotone on hyperplanes; hence, the proximal paths are exact Pareto efficient portfolios.

5.1.2 Performance of Proximal Path Portfolios: Visualization in the Objective Space.

Next, we test the performance of the proximal path generated portfolios against the exact Pareto efficient portfolios, as measured by the worst-case return and nominal case return.

Experiment setup. We construct approximate Pareto-efficient portfolios with proximal paths. We consider two general feasible regions $\Delta^{n}=\{x\in\mathbb{R}^{n}:\langle\mathbf{1},x\rangle=1,x\geq 0\}$ and the feasible region of Markowitz++ model as introduced by Boyd et al. (2024). We first construct portfolios with in-sample historical stock return data ($20$ stocks within S$\&$P 500 from 2021-01-01 to 2023-12-30), before testing the nominal and worst-case returns of each portfolio on out-of-sample stock return data (same stocks from 2024-01-01 to 2024-08-01). As the benchmark, we first construct exact Pareto efficient portfolios by solving problem (10) exactly multiple times each under a different $r$. Then we run a single pass of PPM to generate approximate Pareto efficient robust portfolios, i.e., we use the minimum-variance (most robust) portfolio with $r=\infty$ to initialize the proximal path for solving the nominal portfolio optimization problem with $r=0$, the proximal path generates approximate Pareto-efficient portfolios.

Results. The experiment results are presented in Figure 5, the proximal path generated approximate Pareto-efficient portfolios match closely the exact Pareto-efficient portfolios in performance as measured by the nominal and the worst-case return rates.

5.2 Second Experiment: Obtaining Multiple Robust Solutions in Deep Learning

Adversarial training as robust optimization. The goal in adversarially robust deep learning is to learn networks that are robust against adversarial attacks (i.e., perturbations on the input examples that aim to deteriorate the accuracy of classifiers). A common strategy to robustify networks is adversarial training, which can be formulated as the following robust optimization problem (Madry et al. 2018),

[TABLE]

where $\mathcal{D}$ is the distribution generating pairs of examples $x\in\mathbb{R}^{d}$ and labels $y\in[c]$, $f_{\theta}$ is a neural network parameterized by $\theta$, $\xi$ is the perturbation/attack on the input data, within a perturbation set $\Xi(r,\mathcal{V})$, and $\ell$ is the lost function. Standard adversarial training methods (Madry et al. 2018, Wong et al. 2020) approximately solve problem (11). If adversarial training under a fixed $r$ can be computed in $O(T_{\mathrm{at}})$ time, generating $n$ adversarially robust models under varying $r$ (i.e., a robust path) takes $O(nT_{\mathrm{at}})$.

As an adaptation of Algorithm 1 for problem (11), we propose Algorithm 2 to compute an approximate robust path of problem (11) in only two algorithmic passes: a single adversarial training followed by a single standard training with approximate PPM initialized with $x_{\mathrm{R}}$; finally the approximate PPM iterates are an approximate robust path of problem (11). Assume one step of an approximate PPM step in standard training costs $O(T_{\mathrm{ppm}})$, generating $n$ adversarially robust models under varying $r$ via algorithm 2 takes $O(T_{\mathrm{at}}+nT_{\mathrm{ppm}})$.

Experiment result. We set up problem (11) as training robust image classifiers with the CIFAR10 dataset and a PreAct ResNet18 architecture. As the benchmark for algorithm 2, we first adversarially train networks with the state-of-the-art FGSM (Wong et al. 2020) under different perturbation set radii, $r$. Then we run Algorithm 2 to generate another approximate robust path of problem (11). The performance (clean accuracy v.s. adversarial accuracy) of Algorithm 2 computed robust paths is comparable to that of FGSM (Figure 6). At the same time, the computation cost of our Algorithm 2 is significantly lower than that of FGSM (Table 1).

6 Conclusion

We discuss three pieces of future research that are out of the scope of this paper, but are immediate and important directions for possible future papers:

• •

Nonlinear objective functions. Although our theory is on robust optimization problems with linear objectives as per its closest robust optimization literature of Bertsimas and Sim (2004), Iancu and Trichakis (2014), Chuong and Thanh Tinh (2025), our results can be generalized to convex robust optimization problems via an epigraph reformulation, where the geometrical view of Theorem 3.6 and all subsequent results can be restated through epigraphical projections. In addition, as demonstrated in Section 5.2, the computational speedup of our framework is significant in recovering the robust path of nonlinear (nonconvex) robust optimization problems, where the exact proximal point method is replaced by its computationally cheaper approximations. This also leads to the next research direction.

• •

Approximate proximal point method: trade-off approximation quality with computation cost. In the practical use of Algorithm 1, especially when applied to nonlinear robust optimization problems, the exact proximal point steps can be replaced by its computationally cheaper approximations (e.g., projected gradient descent (Parikh et al. 2014), extra-gradient, optimistic gradient (Mokhtari et al. 2020)) to trade a higher approximation error to the robust path for a lower computational cost. The approximation error between the proximal path and its cheaper algorithmic approximations can be combined directly with our approximation error between the robust path and the proximal path.

• •

Automated tuning of robustness and efficiency trade-off. Our Algorithm 1 recovers the entire robust path. What remains to be answered is which robust solution from the robust path should be deployed? On a high level, our results take the crucial step towards addressing this problem by reducing the high-dimensional solution search space to the one-dimensional robust path recoverable via a proximal path. A natural follow-up work is the algorithmic automation for selecting the robust solution with an appropriate robustness and efficiency trade-off on the now reduced search space of the robust path attainable via a proximal path.

7 Proof of Lemma 2.8

Denote $x^{+}=\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{C})}(x)$ and $x^{++}=\Pi^{\varphi}_{\mathcal{C}}\circ\Pi^{\varphi}_{\mathrm{Aff}(\mathcal{C})}(x)$. By Lemma 2.7,

[TABLE]

and

[TABLE]

Given $x^{++}\in\mathcal{C}$ and (12) we also have

[TABLE]

Adding (12) and (13) we have for all $z\in\mathcal{C}$,

[TABLE]

where the second inequality is due to (14). Finally by Lemma 2.7, we have $x^{++}=\Pi^{\varphi}_{\mathcal{C}}(x)$. \Halmos

8 Proofs of Lemmas 3.4 and 3.5

Proof 8.1

Proof of Lemma 3.4. Given $\Xi=\Xi(r,\mathcal{V})=\left\{\xi\in\mathbb{R}^{n}:\|\xi\|_{\mathcal{V}}\leq r\right\},$

[TABLE]

The last equality is due to Assumption 3.2: $0\in\mathrm{int}(\mathcal{V})$ and Rockafellar (1970) Theorem 14.5. \Halmos

Proof 8.2

Proof of Lemma 3.5. For any $\omega>0$, by the definition of $x_{\mathrm{R}}^{\prime}(\omega,\mathcal{V})$,

[TABLE]

given $r(\omega)=\omega\nabla g(\|x_{\mathrm{R}}^{\prime}\left(\omega,\mathcal{V})\|_{\mathcal{V}^{\circ}}\right)$, we have

[TABLE]

By Lemma 3.4,

[TABLE]

\Halmos

9 $\kappa$-Expansiveness Example

When $\varphi=\frac{1}{2}\|x\|^{2}_{2}$, the usual Euclidean projection $\Pi^{\varphi}_{\mathcal{S}}$ is 1-expansive. More generally, for smooth and strongly convex $\varphi$, it is easy to verify the following result.

Definition 9.1

$\varphi$* is $L\text{-smooth}$ and $\mu\text{-strongly convex}$ w.r.t. some norm $\|\cdot\|$ if*

[TABLE]

Proposition 9.2

Assume $\varphi$ is $L\text{-smooth}$ and $\mu\text{-strongly convex}$ w.r.t. some norm $\|\cdot\|$, then the induce Bregman projection $\Pi^{\varphi}_{\mathcal{S}}$ is $\kappa$-expansive with $\kappa=\left(\frac{L}{\mu}\right)^{3}$.

Proof 9.3

Proof. We begin by proving $D_{\varphi}\left(\Pi^{\varphi}_{\mathcal{S}}(x),\Pi^{\varphi}_{\mathcal{S}}(y)\right)\leq(L/\mu)^{3}\cdot D_{\varphi}\left(x,y\right),\ \forall x,y\in\mathrm{int}(\mathrm{dom}(\varphi))$:

Denote $x^{+}=\Pi^{\varphi}_{\mathcal{X}}(x)$ and $y^{+}=\Pi^{\varphi}_{\mathcal{X}}(y)$, by Lemma 2.7, we have

[TABLE]

and

[TABLE]

Given $x^{+}\in\mathcal{X}$ and $y^{+}\in\mathcal{X}$,

[TABLE]

and

[TABLE]

Combining (17) and (18),

[TABLE]

By the strong convexity of $\varphi$,

[TABLE]

By the smoothness of $\varphi$,

[TABLE]

By the definition of smoothness and strong convexity,

[TABLE]

Together with (20),

[TABLE]

Next, we prove that denote $d=y-x$, $D_{\varphi}\left(\Pi^{\varphi}_{\mathcal{S}+d}(x),\Pi^{\varphi}_{\mathcal{S}}(x)\right)\leq(L/\mu)^{3}\cdot D_{\varphi}\left(x,y\right),\ \forall x,y\in\mathrm{int}(\mathrm{dom}(\varphi))$: Denote $x^{++}=\Pi^{\varphi}_{\mathcal{X}+d}(x)$ and $x^{+}=\Pi^{\varphi}_{\mathcal{X}}(x)$, by lemma 2.7,

[TABLE]

and

[TABLE]

Given $x^{+}\in\mathcal{X}$ and $x^{++}\in\mathcal{X}+d$, or equivalently $x^{+}+d\in\mathcal{X}+d$ and $x^{++}-d\in\mathcal{X}$, we have

[TABLE]

and

[TABLE]

Combining (24) and (25),

[TABLE]

By the strong convexity of $\varphi$,

[TABLE]

By the smoothness of $\varphi$,

[TABLE]

By the definition of smoothness and strong convexity,

[TABLE]

Together with (26),

[TABLE]

\Halmos

10 Proof of Proposition 4.9

We begin by proving (C1) is a sufficient condition for two paths to coincide: $\{x_{k}\}$ is monotone on $\mathcal{X}$ $\Rightarrow$ $x_{k+1}=x_{\mathrm{{CP}}}\left(\omega_{k+1}\right),\ \forall k\in[0,K],$ where $\omega_{k}=\left(\sum_{j=0}^{k-1}\lambda_{j}^{-1}\right)^{-1}$. We proceed with a proof by induction: For $k=0$, by Definition 3.1 and 3.2, we have $x_{1}=x_{\mathrm{{CP}}}\left(\lambda_{0}\right)$. Next, we prove $x_{k}=x_{\mathrm{{CP}}}(\omega_{k})$ and (C1) $\Rightarrow$ $x_{k+1}=x_{\mathrm{{CP}}}(\omega_{k+1})$. By Theorem 18.2. of Rockafellar (1970), for any $x\in\mathcal{X}$, $\exists\mathcal{F}$ which is a face of $\mathcal{X}$ such that $x\in\mathrm{ri}(\mathcal{F})$. Under this guarantee, let $x_{k}\in\mathrm{ri}(\mathcal{F}^{\prime})$, where $\mathcal{F}^{\prime}$ is a face of $\mathcal{X}$. Together with $x_{k}=x_{\mathrm{{CP}}}\left(\omega_{k}\right)$: $\left\langle a_{0}+\omega_{k}\left(\nabla\varphi(x_{k})-\nabla\varphi(x_{0})\right),x-x_{k}\right\rangle\geq 0,\ \forall x\in\mathcal{X},$ we have: $\left\langle a_{0}+\lambda_{0}\left(\nabla\varphi(x_{k})-\nabla\varphi(x_{0})\right),x-x_{k}\right\rangle=0,\ \forall x\in\mathcal{F}^{\prime}.$ By the monotonicity of $\{x_{k}\}$ on $\mathcal{X}$, $x_{k+1}\in\mathcal{F}^{\prime}$, therefore: $\left\langle a_{0}+\lambda_{0}\left(\nabla\varphi(x_{k})-\nabla\varphi(x_{0})\right),x_{k+1}-x_{k}\right\rangle=0,$ by Claim 5, we have $x_{k+1}=x_{\mathrm{{CP}}}(\omega_{k+1})$, where $\omega_{k}=\left(\sum_{j=0}^{k-1}\lambda_{j}^{-1}\right)^{-1}.$

For C2 and C3, it is trivial that if $\mathcal{X}$ is an affine subspace $\{x\in\mathbb{R}^{n}:Ax=b\}$ or the entire vector space $\mathbb{R}^{n}$, then $\{x_{k}\}$ must be monotone on $\mathcal{X}$.

Claim 5

Assume $\mathcal{V}$ satisfies Assumption 3.2, and $\varphi$ satisfies Assumption 3.2. Let $\{x_{k}\}$ be a proximal path initialized by $x_{0}$ associated with the step-size sequence $\{\lambda_{k}\}$. Let $\{x_{\mathrm{{CP}}}(\omega_{k})\}$ be the central path initialized also at $x_{0}$. If $\{x_{k}\}$ satisfies $\left\langle a_{0}+\omega_{k}\left(\nabla\varphi(x_{k})-\nabla\varphi(x_{0})\right),x_{k+1}-x_{k}\right\rangle=0,\ \forall k\in[1,K],$ then $x_{k+1}=x_{\mathrm{{CP}}}\left(\omega_{k+1}\right),\ \forall k\in[0,K],$ where $\omega_{k}=\left(\sum_{j=0}^{k-1}\lambda_{j}^{-1}\right)^{-1}.$

Proof 10.1

Proof. For $k=0$, we show $x_{1}=x_{\mathrm{{CP}}}\left(\lambda_{0}\right)$. By the variational inequality (V.I.) definition of $x_{1}$ as the proximal point update from $x_{0}$, we have

[TABLE]

which is also precisely the V.I. definition of for $x_{\mathrm{{CP}}}\left(\lambda_{0}\right)$, therefore $x_{1}=x_{\mathrm{{CP}}}\left(\lambda_{0}\right)$.

For $k\in[1,K]$, we provide a proof by induction. We begin by proving for $k=1$, if $\left\langle a_{0}+\lambda_{0}\left(\nabla\varphi(x_{1})-\nabla\varphi(x_{0})\right),x_{2}-x_{1}\right\rangle=0$, then $x_{2}=x_{\mathrm{{CP}}}((\lambda_{0}^{-1}+\lambda_{1}^{-1})^{-1})$: Similar to the V.I. definition of $x_{1}$ as the proximal point update from $x_{0}$, $\left\langle a_{0}+\lambda_{0}\left(\nabla\varphi(x_{1})-\nabla\varphi(x_{0})\right),x-x_{1}\right\rangle\geq 0,\ \forall x\in\mathcal{X},$ given $x_{2}$ is the proximal point update from $x_{1}$, we have

[TABLE]

Or equivalently,

[TABLE]

and

[TABLE]

Combining the two inequalities, we have

[TABLE]

Then, if the following equality holds: $\left\langle a_{0}+\lambda_{0}\left(\nabla\varphi(x_{1})-\nabla\varphi(x_{0})\right),x_{2}-x_{1}\right\rangle=0,$ or equivalently

[TABLE]

we have

[TABLE]

which simplifies to

[TABLE]

By definition, $x_{2}=x_{\mathrm{{CP}}}((\lambda_{0}^{-1}+\lambda_{1}^{-1})^{-1})$.

To finish the induction, we prove for any $k\in[2,K]$, if the following equality holds, $\left\langle a_{0}+\omega_{k}\left(\nabla\varphi(x_{k})-\nabla\varphi(x_{0})\right),x_{k+1}-x_{k}\right\rangle=0$ and assume $x_{k}=x_{\mathrm{{CP}}}\left(\omega_{k}\right)$, then we have $x_{k+1}=x_{\mathrm{{CP}}}\left(\omega_{k+1}\right)$: By the V.I. definition of $x_{k}=x_{\mathrm{{CP}}}\left(\omega_{k}\right)$:

[TABLE]

given $x_{k+1}$ is the proximal point update from $x_{k}$, we have

[TABLE]

Or equivalently,

[TABLE]

and

[TABLE]

Combining the two inequalities, we have

[TABLE]

then, if the following equality holds:

[TABLE]

or equivalently

[TABLE]

we have

[TABLE]

which simplifies to

[TABLE]

By definition, $x_{k+1}=x_{\mathrm{{CP}}}((\omega_{k}^{-1}+\lambda_{k}^{-1})^{-1})=x_{\mathrm{{CP}}}(\omega_{k+1})$. \Halmos

11 Proof of Proposition 4.11

Under Assumption 3.2, $g$ is essentially strict convex with $\nabla g(0)=0$ $\Rightarrow$ $g$ is monotonically increasing over $\mathbb{R}_{+}$, as a result $\mathcal{X}=\{x\in\mathbb{R}^{n}:\ \|x\|_{\mathcal{U}}\leq l\}=\{x\in\mathbb{R}^{n}:\ \varphi(x)\leq g(l)\}$, where $\varphi(x)=g\circ\|x\|_{\mathcal{U}}$. In addition, given the uncertainty set is designed to be $\mathcal{U}^{\circ}$, $\varphi(x)=g\circ\|x\|_{\mathcal{U}}=g\circ\|x\|_{\mathcal{U}^{\circ\circ}}$ is also the distance-generating function inducing the optimization path approximations of the robust path, namely $\{x_{k}\}$ and $\{x_{\mathrm{{CP}}}(\omega)\}$. By Definition 3.2, for any $\omega\geq 0$, $x_{\mathrm{{CP}}}(\omega)=\argmin_{x\in\mathbb{R}^{n}}\{\langle a_{0},x\rangle+\omega D_{\varphi}(x,x_{0}):\ \varphi(x)\leq g(l)\}.$ Applying the KKT condition, we obtain

[TABLE]

where $\theta_{\omega}=\left(1+\mu\omega^{-1}\right)^{-1}$ for some Lagrangian dual variable $\mu=\mu(\omega)>0$ such that $\varphi(x_{\mathrm{{CP}}}(\omega))=g(l)$. Geometrically, $\theta_{\omega}$ can be interpreted as a rescaling factor of the dual space vector $\left(\nabla\varphi(x_{0})-\omega^{-1}a_{0}\right)$ such that $\varphi(x_{\mathrm{{CP}}}(\omega))=g(l)$, leading to $x_{\mathrm{{CP}}}(\omega)=\Pi^{\varphi}_{\{x\in\mathbb{R}^{n}:\ \varphi(x)\leq g(l)\}}\left(\nabla\varphi^{*}\left(\theta_{\omega}\left(\nabla\varphi(x_{0})-\omega^{-1}a_{0}\right)\right)\right)=\nabla\varphi^{*}\left(\theta_{\omega}\left(\nabla\varphi(x_{0})-\omega^{-1}a_{0}\right)\right)$.

Apply a similar procedure for the $\{x_{k}\}$ yields for any $k$,

[TABLE]

where $\theta_{\lambda_{k}}=\left(1+\mu\lambda_{k}^{-1}\right)^{-1}$ for some Lagrangian dual variable $\mu=\mu(\lambda_{k})>0$ such that $\varphi(x_{k})=g(l)$.

Given the above characterization of $\{x_{\mathrm{{CP}}}(\omega)\}$ and $\{x_{k}\}$, we now prove $\{x_{k}\}\subset\{x_{\mathrm{{CP}}}\left(\omega\right):\omega\in[0,\infty)\}$ by showing for any $x_{k}\in\{x_{k}\}$, there exists a $x_{\mathrm{{CP}}}\left(\omega_{k}\right)\in\{x_{\mathrm{{CP}}}\left(\omega\right):\omega\in[0,\infty)\}$ such that $x_{k}=x_{\mathrm{{CP}}}\left(\omega_{k}\right)$. We proceed with a proof by induction.

For $k=1$: by Definition 3.1 and 3.2, we have $x_{1}=x_{\mathrm{{CP}}}(\lambda_{0})$.

For $k>1$: assume $x_{k}=x_{\mathrm{{CP}}}(\omega_{k})$, in the dual space, w.l.o.g., assume $\varphi(x_{k})<g(l)$, we have

[TABLE]

Next, given $x_{k+1}$ as the proximal point update of $x_{k}$, w.l.o.g., assume $\varphi(x_{k+1})=g(l)$ yields

[TABLE]

By invoking again Lemma 2.3 and mapping both the LHS and RHS back to the primal space, we have $x_{k+1}\in\{x_{\mathrm{{CP}}}\left(\omega\right):\omega\in[0,\infty)\}$.

12 Proof of Theorem 4.13

We begin by constructing for each $i\in[I]$, two auxiliary central paths that mirror exactly $\left\{x_{k}:k\in\left[\underline{k}^{(i)}+1,\overline{k}^{(i)}\right]\right\}$ and $\left\{x_{\mathrm{{CP}}}\left(\upsilon_{k}^{-1};x_{0}\right):k\in\left[\underline{k}^{(i)}+1,\overline{k}^{(i)}\right]\right\}$ respectively. The distance between the two original sequences can be analyzed equivalently via the distance between the two auxiliary sequences.

Step One: We first construct the auxiliary central path for $\left\{x_{k}:k\in\left[\underline{k}^{(i)}+1,\overline{k}^{(i)}\right]\right\}$. By Definition 4.8, $\left\{x_{k}:k\in\left[\underline{k}^{(i)}+1,\overline{k}^{(i)}\right]\right\}$ is monotone on $\mathrm{ri}(\mathcal{F}_{i})$, consequently by Proposition 4.9 we have

[TABLE]

where $\upsilon_{k}^{\prime}=\sum_{j=\underline{k}^{(i)}}^{k-1}\lambda_{j}^{-1}$. It is helpful to consider its Bregman projection reformulation due to Theorem 3.6: $x_{\mathrm{{CP}}}\left(\left(\upsilon_{k}^{\prime}\right)^{-1};x_{\underline{k}^{(i)}}\right)=\Pi^{\varphi}_{\mathrm{ri}(\mathcal{F}_{i})}\left(\nabla\varphi^{*}\left(\nabla\varphi\left(x_{\underline{k}^{(i)}}\right)-\upsilon_{k}^{\prime}a_{0}\right)\right)$.

Step Two: Next we construct the auxiliary central path for $\left\{x_{\mathrm{{CP}}}\left(\upsilon_{k}^{-1};x_{0}\right):k\in\left[\underline{k}^{(i)}+1,\overline{k}^{(i)}\right]\right\}$ where $\upsilon_{k}=\underline{\upsilon}^{(i)}+\sum_{j=\underline{k}^{(i)}}^{k-1}\lambda_{j}^{-1}$, and show the two sequences are equivalent. First, we define the following point that is critical:

[TABLE]

By Theorem 3.6 and given $\upsilon_{k}=\underline{\upsilon}^{(i)}+\sum_{j=\underline{k}^{(i)}}^{k-1}\lambda_{j}^{-1}$, $\left\{x_{\mathrm{{CP}}}\left(\upsilon_{k}^{-1};x_{0}\right):k\in\left[\underline{k}^{(i)}+1,\overline{k}^{(i)}\right]\right\}$ has the following Bregman projection reformulation,

[TABLE]

We define its auxiliary sequence as

[TABLE]

where $\upsilon_{k}^{\prime}=\sum_{j=\underline{k}^{(i)}}^{k-1}\lambda_{j}^{-1}$.

By definition, $\Pi^{\varphi}_{\mathcal{X}}\left(\bigstar\right)\in\mathrm{ri}(\mathcal{F}_{i})$, therefore by Claim 6 we have $\Pi^{\varphi}_{\mathrm{ri}(\mathcal{F}_{i})}(\bigstar)=\Pi^{\varphi}_{\mathrm{Aff}(\mathrm{ri}(\mathcal{F}_{i}))}(\bigstar).$ Consequently, by the same argument as in the proof of Corollary 4.5, two sequences are equivalent, and we have

[TABLE]

where $\upsilon_{k}=\underline{\upsilon}^{(i)}+\sum_{j=\underline{k}^{(i)}}^{k-1}\lambda_{j}^{-1}$ and $\upsilon_{k}^{\prime}=\sum_{j=\underline{k}^{(i)}}^{k-1}\lambda_{j}^{-1}$.

Step Three: By the results established in the previous two steps, for each $k\in\left[\underline{k}^{(i)}+1,\overline{k}^{(i)}\right]$ we have

[TABLE]

where the third equality is due to Claim 6 and the inequality is due to Claim 7. \Halmos

Claim 6

Let $\mathcal{X}$ be a closed convex polyhedron, and let $\mathcal{F}$ be a face of $\mathcal{X}$. Additionally, assume $\varphi$ is Legendre, then

[TABLE]

Proof 12.1

Proof of Claim 6. Let $\mathcal{N}_{\mathcal{C}}(p)$ denote the normal cone of a convex set $\mathcal{C}$ at $p\in\mathcal{C}$, by Lemma 2.7, we have

[TABLE]

The result follows by proving the following statement is true:

[TABLE]

We begin by showing $\mathcal{N}_{\mathcal{X}}(y)\subset\mathcal{N}_{\mathrm{ri}(\mathcal{F})}(y)$. Assume $d\in\mathcal{N}_{\mathcal{X}}(y)$, by definition we have

[TABLE]

given $\mathrm{ri}(\mathcal{F})\subset\mathcal{X}$ we have

[TABLE]

hence $d\in\mathcal{N}_{\mathrm{ri}(\mathcal{F})}(y)$.

By the same argument it follows that $\mathcal{N}_{\mathrm{Aff}(\mathrm{ri}(\mathcal{F}))}(y)\subset\mathcal{N}_{\mathrm{ri}(\mathcal{F})}(y)$. It remains to show $\mathcal{N}_{\mathrm{ri}(\mathcal{F})}(y)\subset\mathcal{N}_{\mathrm{Aff}(\mathrm{ri}(\mathcal{F}))}(y)$: Assume $g\in\mathcal{N}_{\mathrm{ri}(\mathcal{F})}(y)$, by definition we have

[TABLE]

For any $w\in\mathrm{Aff}(\mathrm{ri}(\mathcal{F}))$, , there exist some $\alpha\geq 0$ such that

[TABLE]

consequently,

[TABLE]

hence $g\in\mathcal{N}_{\mathrm{Aff}(\mathrm{ri}(\mathcal{F}))}(y)$. We have shown,

[TABLE]

To finish the proof, given

[TABLE]

by $\mathcal{N}_{\mathcal{X}}(y)\subset\mathcal{N}_{\mathrm{ri}(\mathcal{F})}(y)=\mathcal{N}_{\mathrm{Aff}(\mathrm{ri}(\mathcal{F}))}(y)$, we have

[TABLE]

Or equivalently,

[TABLE]

\Halmos

Claim 7

Let $\mathcal{A}\subset\mathbb{R}^{n}$ be a affine subspace, let $x,y\in\mathcal{A}$ and let $\varphi$ be a Legendre function whose Bregman projection is $\kappa-\text{expansive}$, then for any $c\in\mathbb{R}^{n}$,

[TABLE]

Proof 12.2

Proof of Claim 7. Denote $\mathcal{A}=\mathcal{L}+x$, where $\mathcal{L}=\mathcal{A}-x$ is a linear subspace. For any $\upsilon\in[0,\infty)$, by Lemma 2.10, we have

[TABLE]

and

[TABLE]

With the above results, next we map the pair $\Pi^{\varphi}_{\mathcal{A}}\left(\nabla\varphi^{*}\left(\nabla\varphi(x)-\upsilon c\right)\right)$ and $\Pi^{\varphi}_{\mathcal{A}}\left(\nabla\varphi^{*}\left(\nabla\varphi(y)-\upsilon c\right)\right)$ to the dual space before mapping them back to the primal space. For any $\upsilon\in[0,\infty)$,

[TABLE]

where the first equality is due to Bauschke et al. (1997) Theorem 3.7(v), the inequality is by Definition 4.1 and the third equality is again due to Bauschke et al. (1997) Theorem 3.7(v). \Halmos