Safe Convex Learning under Uncertain Constraints

TL;DR

This paper introduces a robust Frank-Wolfe algorithm for convex optimization with uncertain linear constraints, ensuring feasibility and convergence in safety-critical applications like medicine and robotics.

Contribution

A novel Frank-Wolfe variant that handles uncertain constraints with high-probability guarantees and convergence analysis.

Findings

Guarantees feasibility of all iterates.

Achieves convergence rate under uncertainty.

Applicable to safety-critical domains.

Abstract

We address the problem of minimizing a convex smooth function $f(x)$ over a compact polyhedral set $D$ given a stochastic zeroth-order constraint feedback model. This problem arises in safety-critical machine learning applications, such as personalized medicine and robotics. In such cases, one needs to ensure constraints are satisfied while exploring the decision space to find optimum of the loss function. We propose a new variant of the Frank-Wolfe algorithm, which applies to the case of uncertain linear constraints. Using robust optimization, we provide the convergence rate of the algorithm while guaranteeing feasibility of all iterates, with high probability.

Figures

Equations

$$\underset{x\in {\mathbb{R}}^d}{\min }\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}f(x)$$

$$\text{subject to }{f}_i(x)\le 0\text{leavevmode}\text{leavevmode}\forall i=1,\dots ,m,$$

$$\underset{x\in {\mathbb{R}}^d}{\min }\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}f(x)$$

$$\text{subject to }Ax-b\le 0.$$

$$\mathbb{P}\{f(x_T)-f(x_{\ast })\le ϵ\}\ge 1-\delta .$$

$$\mathbb{P}\{A{x}_t-b\le 0,0\le t\le T\}\ge 1-\delta .$$

$${\hat{s}}_t=\arg \underset{s\in {\hat{D}}_t}{\min }⟨\nabla f(x_t),s⟩.$$

$$\forall \lambda \in \mathbb{R}\text{leavevmode}\forall i=1,\dots ,m\text{leavevmode}\forall j\le N_t\text{leavevmode}\mathbb{E}\left[e^{\lambda {\eta }_j^i}\right]\le \text{exp}\left(\frac{{\lambda }^2{\sigma }^2}{2}\right).$$

$${\hat{\beta }}_t=[{\hat{A}}_t,{\hat{b}}_t{]}^T=[{\bar{X}}_t^T{\bar{X}}_t{]}^{-1}{\bar{X}}_t^T{Y}_t.$$

$${\hat{g}}_t$$

$$s_t=\arg \underset{s\in D}{\min }⟨s,\nabla f(x_t)⟩,\text{leavevmode}\text{leavevmode}{g}_t=\underset{s\in D}{\min }⟨s,\nabla f(x_t)⟩.$$

$$x_{(N_{t-1}+i)}=x_t+e_i{\omega }_0,$$

$$x_{(N_{t-1}+2i)}=x_t-e_i{\omega }_0,i=1,\dots ,d,$$

$${\varphi }^{-1}(\bar{\delta })=\max \left\{\sqrt{128d\log N_t\log \left(\frac{N_t^2}{\bar{\delta }}\right)},\frac{8}{3}\log \frac{N_t^2}{\bar{\delta }}\right\}$$

$$S_t(\bar{\delta })=\{x\in {\mathbb{R}}^d:Ax\le b,\text{leavevmode}\forall [A,b]\in {\mathcal{E}}_t(\bar{\delta })\}.$$

$$\displaystyle S_{t}(\bar{\delta})=\Bigg{\{}x\in$$

$$\displaystyle+\phi^{-1}(\bar{\delta}/m)\left\|{\Sigma_{t}^{1/2}}\begin{bmatrix}x\\ -1\end{bmatrix}\right\|\leq 0\Bigg{\}}.$$

$$\mathbb{P}\{x\in D|x\in S_t(\bar{\delta }),\beta \in {\mathcal{E}}_t(\bar{\delta })\}=1.$$

$${\varphi }_{\bar{\delta }}\sqrt{\frac{1}{N_t}+(x_t-{\bar{x}}_t{)}^T{R}_t(x_t-{\bar{x}}_t)}\le \underset{i=1,\dots ,m}{\min }{\epsilon }_t^i,$$

$$C_n$$

$$\text{where }{C}_{\bar{\delta }}$$

$$\mathbb{P}$$

$$\mathbb{P}\{{\mathcal{F}}_T\}$$

$$\ge 1-\sum _{t=0}^T\sum _{i=1}^m\bar{\delta }/m\ge 1-T\bar{\delta }.$$

$$C_f=\underset{\begin{array}{c}x,s\in D,\gamma \in [0,1],\\ y=x+\gamma (s-x)\end{array}}{\sup }\frac{1}{{\gamma }^2}(f(y)-f(x)-⟨y-x,\nabla f(x)⟩).$$

$$\displaystyle\mathbb{P}\Bigg{\{}$$

$$\displaystyle+\frac{\ln{(T+2)}\frac{C_{f}}{2}+\ln\ln(T+2)\frac{C^{\prime}}{2}}{T+2}\Bigg{\}}\geq 1-\delta,$$

$$f(x_t)-f(x_{\ast })\le O\left(\frac{{ϵ}_0+C_f\ln t+{\sum }_{t=1}^T{E}_t}{t}\right),$$

$$\underset{x\in D}{\min }\frac{1}{2}{\Vert x-x^{\prime }\Vert }_2^2,$$

$$S_t(\bar{\delta })$$

$$\displaystyle+\phi^{-1}(\bar{\delta}/m)\sigma\left\|(\bar{X}_{t}^{T}\bar{X}_{t})^{-1/2}\begin{bmatrix}x\\ -1\end{bmatrix}\right\|\leq 0\Bigg{\}}.$$

$$({\bar{X}}_t^T{\bar{X}}_t{)}^{-1}$$

$$\displaystyle=\left[\begin{array}[]{cc}R_{t}&R_{t}\bar{x}_{t}\\ \bar{x}_{t}^{T}R_{t}&\frac{1}{N_{t}}+\bar{x}_{t}^{T}R_{t}\bar{x}_{t}\end{array}\right],$$

$$R_t$$

Full text

1 INTRODUCTION

Many optimization tasks in robotics, health sciences, and finance require minimizing a loss function under uncertainties. Most existing stochastic and online optimization approaches proposed to address these tasks assume that the constraints of the corresponding optimization problems are known. These approaches, however, are unacceptable in cases in which the feasible set is itself unknown and is learned online. Optimizing a loss function under such a partially revealed feasible set model is further challenged by the fact that exploration can be made only inside the feasible set due to safety reasons. Hence, one needs to carefully choose actions to ensure feasibility of each iterate with high probability while learning the optimal solution. In the machine learning community, this problem is known as safe learning.

Safe learning is receiving increasing attention due to the increasingly widespread deployment of machine learning in safety-critical tasks. An example arises in personalized medicine, where physicians may choose from a large set of therapies. The effects of different therapies on the patient are initially unknown, and can only be determined through clinical trials. Free exploration however is not possible since some therapies might cause discomfort or even physical harm (Sui et al.,, 2015). Similar challenges arise in designing control algorithms for robots, which have to navigate unexplored terrains or interact with humans (Cassandra et al.,, 1996; Koenig and Simmons,, 1996). In these scenarios, robots need to learn the best tuning for their controllers or optimize their trajectories based on risky experimental interactions with the partially known environment.

We address the problem of safe learning given a convex loss function subject to unknown constraints. Motivated by the aforementioned applications, we assume that the decision maker has access only to noisy observations of the constraints for a chosen action. Our objective is to design an algorithm that sequentially steers the decisions towards the optimum while ensuring safety of the decisions at every step. In this paper, we restrict ourselves to unknown linear constraints. Perhaps surprisingly, there is very little work on safe learning for this rudimentary setup. Hence, we consider this as a first step towards developing fundamental understanding and design of efficient algorithms for the more general nonlinear and non-convex setting.

Related work.

There are many optimization algorithms that ensure feasibility of the iterates, assuming a known constraint function. The most basic ones are projected gradient descent (PGD)(Boyd and Vandenberghe,, 2004) and Frank-Wolfe (FW) (Frank and Wolfe,, 1956) (also known as conditional gradient). These methods require exact knowledge of the constraints or at least a projection oracle or an exact linear programming (LP) oracle with respect to the constraints. However, as discussed above, this information may not be available in safe learning problems.

Assuming the functional form of the constraints, with parameters drawn from a probability distribution, chance-constraint optimization addresses the problem of optimizing a loss function subject to constraint satisfaction with a sufficiently high probability (El Ghaoui et al.,, 1998). The proposed solution methods for this problem assume either a priori knowledge of the distribution of the parameters (El Ghaoui et al.,, 1998), moments of the distribution (Zymler et al.,, 2013), or a sufficiently large number of samples of the distribution (Calafiore and Campi,, 2005). In contrast, in a safe learning problem, the decision maker does not have access to such information a priori. This information is gathered online and feasibility needs too be ensured while exploring the uncertain decision space.

A recent line of work addresses uncertain constraints in online stochastic optimization (Yu et al.,, 2017; Yu and Neely,, 2016). The work is based on infeasible penalty methods, and thus does not provide guarantees on constraint violation at each iteration. Rather, the methods ensure convergence of the average constraint violation to zero. Similarly, risk-aware contextual bandits and bandits with knapsack constraints (Sun et al.,, 2017; Mahdavi et al.,, 2012; Jenatton et al.,, 2015) consider unknown constraint functions with a budget limit. Here, safety refers to ensuring that the total usage of a commodity, e.g., budget for adverts, summed over the sequence of iterates remains below a threshold. Similar to Yu et al., (2017); Yu and Neely, (2016), the above approaches bound average constraint violation, rather than avoiding violation at each iteration. While such a formulation can be well-suited in certain problems such as adverts, it may not be well-suited for safe learning applications discussed above because in this latter case, constraints need to be satisfied at each step.

The problem of safe learning using Gaussian processes (GP) has been proposed in Sui et al., (2015). The SafeOpt algorithm developed in the above work considers minimizing an unknown loss function iteratively, while ensuring that the loss of each iterate is above a required threshold. Given actively taken measurements of the loss, the initial estimate of the feasible set is incrementally enlarged through exploration and considering certain regularities of GP kernels. This framework is extended to multiple constraints and experimentally validated on robotic platforms by Berkenkamp et al., (2016). Safe GP learning is powerful as it can address general non-convex problems. Nevertheless, due to this generality, current approaches do not scale well with the problem dimension. This motivates our work of developing efficient safe learning algorithms for the case of convex loss functions and constraints.

Our contributions.

We propose a novel algorithm for safe active learning, given a smooth convex objective and a set of unknown linear constraints with noisy oracle information. Given a confidence level $1-\delta$ and accuracy $\epsilon$, we prove that after $\tilde{O}\left(\frac{1}{\epsilon}\right)$ iterations and $\tilde{O}\left(\frac{d^{3}\ln 1/\delta}{\epsilon^{2}}\right)$ constraints measurements, the final point is an $\epsilon$-accurate solution with probability $1-\delta$ (cf. Theorem 2). By $\tilde{O}(\cdot)$ we denote $O(\cdot)$ up to a logarithmic multiplicative factor. Furthermore, we ensure feasibility for the trajectory of the iterates with probability at least $1-\delta$ (cf. Theorem 1). While in this paper we mainly focus on exact first-order oracles for the objective function, we discuss extensions to stochastic oracles in Section 5.

The core idea of our algorithm is to combine a first-order feasible optimization approach with the robust optimization technique. Specifically, our algorithm is based on the Frank-Wolfe (FW) method. In each iteration, it solves an uncertain linear program based on estimates of the constraints and uses this solution to define the step direction. The safety of the iterates is ensured with high probability by refining the confidence set of the unknown parameters iteratively. We emphasize that while we use robust optimization (Ben-Tal and Nemirovski,, 1998, 1999, 2000), our problem formulation is different than that of a classical robust optimization. Specifically, we consider gathering information online about the uncertainty, whereas the robust optimization works assume one-shot knowledge of uncertainties. We numerically evaluate the performance of the proposed algorithm in Section 6 and compare its performance with a one-shot robust optimization approach.

Notations

Let $I^{d}\in\mathbb{R}^{d\times d}$ denote identity matrix, $e_{i}\in\mathbb{R}^{d}$ the unit vector corresponding to the $i$-th coordinate. Let $\|\cdot\|$ denote the Euclidean norm for vectors and the spectral norm for matrices. The ball of radius $r$ centered at a point $x_{0}\in\mathbb{R}^{d}$ is denoted by $B^{d}(x_{0},r)=\{x\in\mathbb{R}^{d}|\|x-x_{0}\|\leq r\}$.

2 PROBLEM FORMULATION

The problem of safe learning in its most general form can be defined as a constrained optimization problem

[TABLE]

where the objective function $f:\mathbb{R}^{d}\rightarrow\mathbb{R}$ and the constraints $f_{i}:\mathbb{R}^{d}\rightarrow\mathbb{R}$ are unknown, and can only be accessed at feasible points $x$. A possibly noisy oracle provides access to the values of these functions or their gradients at any queried feasible $x\in\mathbb{R}^{d}$. The objective is to design an iterative algorithm that chooses the query points to ensure feasibility at each round while progressing towards the optimum. To address this goal, we need to define the oracle more precisely and make some regularity assumptions on the functions.

In this paper, we consider an instance of the safe learning problem in which the objective $f$ is convex and $M$-Lipschitz continuous, that is, $|f(x)-f(y)|\leq M\|x-y\|\leavevmode\nobreak\ \forall x,y\in D$, where $D$ is the feasible set. Furthermore, we assume $f$ is $L$-smooth, that is, $f$ has $L$-Lipschitz continuous gradients in $D$, $\|\nabla f(x)-\nabla f(y)\|\leq L\|x-y\|,\leavevmode\nobreak\ \forall x,y\in D$. We assume access to the gradients of the objective function, $\nabla f(x)$, at any feasible query point $x\in D$. We furthermore assume that constraints are known to be linear, $f_{i}(x)=[a^{i}]^{T}x-b^{i}$ for $i=1,\dots,m$. Hence, letting $A\in\mathbb{R}^{m\times d}$ denote the matrix with rows defined by $[a^{i}]^{T}$, the problem is given by

[TABLE]

We assume that the feasible set $D=\{x\in\mathbb{R}^{d}:Ax-b\leq 0\}$ is a compact polytope with non-empty interior. Denote by $\Gamma$ the diameter of the set $D$, $\Gamma=\max_{x,y\in D}\|x-y\|$. Furthermore, let $\Gamma_{0}$ be the radius of the smallest ball centered at 0 such that $D\in B(0,\Gamma_{0})$, namely, $\Gamma_{0}=\max_{x\in D}\|x\|$.

If $A$ and $b$ are known, (1) can be solved efficiently by off-the-shelf first-order convex optimization algorithms. We however, consider the case in which $A$ and $b$ are unknown and can be accessed through an oracle. Specifically, we assume the constraints can be evaluated at any point that lies within a ball of radius $\omega_{o}$ of the feasible set. These evaluations are corrupted by sub-Gaussian noise. Hence, we have access to $y(x)=Ax-b+\eta$ for any $x$ such that $B^{d}(x,\omega_{0})\cap D\neq\emptyset$, where $\eta$ are sub-Gaussian. If in the problem setting having all the measurements inside the feasible set is critical, we can artificially shrink the set $D$ by the value $\omega_{0}$ from the boundaries. This can be achieved by tightening the constraints $[a^{i}]^{T}x\leq b^{i}$ with setting the measurements $\hat{y}^{i}=y^{i}-\mathcal{\kappa}=[a^{i}]^{T}x-b^{i}+\eta-\mathcal{\kappa}^{i}$, with $\mathcal{\kappa}^{i}\geq L^{i}_{A}\omega_{0}$, where $L^{i}_{A}$ is an upper bound on $\|a^{i}\|$.

The scope of the present paper is to design an algorithm which, starting from a feasible point $x_{0}\in D$, converges to an optimal solution $x_{*}$ with a required accuracy $\epsilon$ and a required confidence $1-\delta$ after $T$ steps, that is,

[TABLE]

Since the constraint set $D$ is unknown and revealed through a noisy oracle, we can at the very best ensure to remain inside the feasible set with sufficiently high probability. Hence, we require that the updates of the method are not violating the true constraints with the same required confidence level of $1-\delta$, that is,

[TABLE]

Some words on the choices of the optimization and oracle above are in order. First, the setting of linear constraints can be restrictive for some real-world problems. Nevertheless, understanding the linear setup is often the first step in addressing more challenging formulations. Second, having a noisy first-order or a zeroth-order oracle for the objective function is more realistic for several safe learning problems. Optimization under such oracle models have been deeply explored for the case in which the constraint set is known. Hence, the main novelty and challenge in safe learning is ensuring feasibility of the iterates despite uncertain and incrementally revealed constraint values. We discuss how the proposed algorithm can be generalized to stochastic oracle models for objective in Section 5.

3 THE SFW ALGORITHM

We propose a variant of the Frank-Wolfe algorithm where we explicitly take into account the uncertainty about the feasible set $D$, referred to as Safe Frank-Wolfe (SFW). The algorithm can be summarized as follows. Starting with a feasible point $x_{0}\in D$, at each iteration $t=0,\dots,T$ we generate a number $n_{t}$ of query points and obtain noisy measurements of the constraint functions at these points. Using linear regression, we obtain an estimate $\hat{D}_{t}$ of the feasible set based on the history of obtained measurements. The algorithm then uses $\hat{D}_{t}$ to obtain a direction $\hat{s}_{t}$ by solving the estimated Direction Finding Subproblem (DFS)

[TABLE]

The next iterate is then given by $x_{t+1}=x_{t}+\gamma_{t}(\hat{s}_{t}-x_{t})$, according to a chosen step-size $\gamma_{t}$. Below, we further describe each step of the proposed algorithm.

Taking Measurements.

During each iteration $t$ of the algorithm, we first make measurements at $n_{t}$ number of points $x_{(j)}$ within distance $\omega_{0}$ of $x_{t}$ in $d$ linearly independent directions. The number $n_{t}$ needs to satisfy a lower bound as a function of the input data $\delta$, $T$, to ensure safety. This bound is provided in Theorem 1. Denote by $X_{t}=[x_{(1)},\ldots,x_{(N_{t})}]^{T}\in\mathbb{R}^{N_{t}\times d}$ and by $N_{t}=\sum_{k=0}^{t}n_{k}$, the total number of available measurements at iteration $t$. Combining all measurements taken up to iteration $t$ we have the following information about the constraints $y^{i}=X_{t}a^{i}-b^{i}\textbf{1}+\eta^{i},\,i=1,\ldots,m,$ where $y^{i}\in\mathbb{R}^{N_{t}}$ is the vector of $N_{t}$ measurements of $i$-th constraint, $\eta^{i}=[\eta^{i}_{1},\ldots,\eta^{i}_{N_{t}}]^{T}\in\mathbb{R}^{N_{t}}$ is the vector of errors. The errors $\eta^{i}_{j}$ are independent and $\sigma$-sub-Gaussian, which means

[TABLE]

The sub-Gaussian condition implies that $\mathbb{E}\left[\eta^{i}_{j}\right]=0$ and $\text{Var}\left[\eta^{i}_{j}\right]\leq\sigma^{2}$. An example of $\sigma$-sub-Gaussian $\eta^{i}$ are independent zero-mean Gaussian random variables with variance at most $\sigma^{2}$, or independent bounded zero-mean variables lying in an interval of length at most $2\sigma$. We also denote by $\mathbf{1}\in\mathbb{R}^{N_{t}}$ the vector of 1’s. Let us denote by $Y_{t}=[y^{1},\ldots,y^{m}]\in\mathbb{R}^{N_{t}\times m}$ the matrix of corresponding measurements of the constraints.

Estimating Constraints.

Let $\beta^{i}=\begin{bmatrix}[a^{i}]^{T}&b^{i}\end{bmatrix}^{T}\in\mathbb{R}^{d+1}$ denote the vector corresponding to the $i$-th constraint. We refer to $\beta^{i}$ as the true parameter. Let $\bar{X}_{t}=[X_{t},-\textbf{1}]\in\mathbb{R}^{N_{t}\times(d+1)}$ be the extended version of the matrix $X_{t}$. The Least Squares Estimation (LSE) of the constraint parameters at step $t$ is given by

[TABLE]

The covariance matrix of the $\hat{\beta}^{i}_{t}$ is given by $\Sigma_{t}=\sigma^{2}[\bar{X}_{t}^{T}\bar{X}_{t}]^{-1}$. Let $\hat{a}_{t}^{i},\hat{b}_{t}^{i}$ denote the estimates of the corresponding rows of $\hat{\beta}^{i}_{t}$ and $\hat{D}_{t}=\{x\in\mathbb{R}^{d}:\hat{A}_{t}x\leq\hat{b}_{t}\}$ denote the estimated feasible set.

Stopping criteria.

Recall that $\hat{s}_{t}$ is the minimizer of the estimated DFS (4) and let $\hat{g}_{t}$ be its optimal value

[TABLE]

Similarly, let $s_{t}$ denote the minimizer of the DFS under true constraints and ${g}_{t}$ the corresponding optimal value

[TABLE]

From convexity of $f$, we have $f(x_{t})-f(x_{*})\leq g_{t}$. Thus, as discussed in Jaggi, (2013), $g_{t}$ can be taken as a surrogate duality gap and consequently a stopping criterion for the FW algorithm. In our case, the duality gap cannot be computed exactly because the feasible set $D$ is unknown. Nevertheless, for the random variable $E_{t}:=|\hat{g}_{t}-g_{t}|$ describing an error in the gap estimation we can derive a probabilistic upper bound $\bar{E}_{t}(\delta)$, such that $\mathbb{P}\{{E_{t}}\leq\bar{E}_{t}(\delta)\}\geq 1-\delta$ (see Proposition 1 Section 5). It follows that if $\hat{g}_{t}+\bar{E}_{t}(\delta)\leq\epsilon$, then with probability greater than $1-\delta$ we have $f(x_{t})-f(x_{*})\leq\epsilon$. Thus, we use $\hat{g}_{t}+\bar{E}_{t}(\delta)\leq\epsilon$ as a stopping criterion.

Putting the above few steps together, we present the Safe Frank-Wolfe (SFW) in Algorithm 1.

4 SAFETY

In order to ensure safety of the trajectory as per Inequality (3) we ensure that each $x_{t+1}$ generated by the algorithm above remains within the feasible set $D$ with probability $1-\bar{\delta},$ where $\bar{\delta}=\frac{\delta}{T}$. This is achieved using the analysis framework of robust optimization by Bertsimas et al., (2011), Ben-Tal and Nemirovski, (1998). The safety of each iterate, combined with a union bound, enables us to prove the safety of the sequence $\{x_{t}\}_{t=1}^{T}$ with probability $1-\delta=1-\sum_{t=1}^{T}\bar{\delta}$.

Since the LSE’s of the constraint parameters are given by $\hat{\beta}_{t}^{i}=[[\hat{a}^{i}_{t}]^{T},\hat{b}_{t}^{i}]^{T}\in\mathbb{R}^{d+1}$, the confidence set for the vector of true parameters $\beta^{i}$ is given by the following ellipsoid: $\mathcal{E}^{i}_{t}(\bar{\delta})=\left\{z\in R^{d+1}:(\hat{\beta}_{t}^{i}-z)^{T}\Sigma_{t}^{-1}(\hat{\beta}_{t}^{i}-z)\leq\phi^{-1}(\bar{\delta})^{2}\right\},$ where,

[TABLE]

for $\sigma$-sub-Gaussian noise $\eta^{i}$ for $N_{t}e^{-1/16}\geq\bar{\delta}$, (Dani et al.,, 2008) 222In the case when the noise is Gaussian and $X_{t}$ is chosen deterministically, e.g. if all the samples are taken around $x_{0}$, $\phi^{-1}(\bar{\delta})$ is taken as the inverse of Chi-squared cumulative distribution function with $d+1$ degrees of freedom (Draper and Smith,, 2014).. Thus, we have an ellipsoidal uncertainty set centered at $\hat{\beta}_{t}^{i}$, such that $\mathbb{P}\{\beta^{i}\in\mathcal{E}^{i}_{t}(\bar{\delta})\}\geq 1-\bar{\delta}.$ We define the confidence set $\mathcal{E}_{t}(\bar{\delta})$ for all parameters $\beta$ by $\mathcal{E}_{t}(\bar{\delta})=\mathcal{E}^{1}_{t}(\bar{\delta}/m)\times\ldots\times\mathcal{E}^{m}_{t}(\bar{\delta}/m)\subseteq\mathbb{R}^{(d+1)m}.$ The confidence set $\mathcal{E}_{t}(\bar{\delta})$ determines the uncertainty set for constraint parameters $\beta$ with probability $1-\bar{\delta}$. Indeed, $1-\mathbb{P}(\exists i:\,\beta^{i}\notin\mathcal{E}^{i}_{t}(\bar{\delta}/m))=1-\mathbb{P}\{\cup_{i=1}^{m}\{\beta^{i}\notin\mathcal{E}^{i}_{t}(\bar{\delta}/m)\}\}\geq 1-\sum_{i=1}^{m}\bar{\delta}/m=1-\bar{\delta}.$

We define the safety set $S_{t}(\bar{\delta})\subset\mathbb{R}^{d}$ at iteration $t$ as the set of $x\in\mathbb{R}^{d}$ satisfying the constraints with any true parameter $\beta=[A,b]$ in the confidence set:

[TABLE]

Given that for each constraint $\beta^{i}$ our uncertainty set $\mathcal{E}^{i}_{t}(\bar{\delta})$ has an ellipsoidal form, the safety set $S_{t}(\bar{\delta})$ is equivalent to the intersection of a set of second order cone constraints (Ben-Tal et al.,, 2009)

[TABLE]

Fact 1**.**

From the definition of the confidence and safety sets it readily follows that

[TABLE]

Fact 2**.**

The condition $x_{t}\in S_{t}(\bar{\delta})$ is equivalent to

[TABLE]

where $\phi_{\bar{\delta}}=\sigma\phi^{-1}(\bar{\delta}/m)$, $\varepsilon^{i}_{t}=\hat{b}_{t}^{i}-[\hat{a}_{t}^{i}]^{T}x_{t}$, $\bar{x}_{t}=\frac{X_{t}^{T}\mathbf{1}}{N_{t}}$, and R_{t}=\big{[}\sum_{j=1}^{N_{t}}(x_{{(j)}}-\bar{x}_{t})(x_{(j)}-\bar{x}_{t})^{T}\big{]}^{-1}.

The proof of this fact is provided in Appendix A.

To state the main result on safety of each iteration, we need to introduce some notation. For the polytope $D\in\mathbb{R}^{d}$, by an active set $B$ we denote a set of indices of $d$ linearly independent constraints active in a vertex $V\in\mathbb{R}^{d}$ of $D$, i.e., $V=V^{B}=[A^{B}]^{-1}b^{B}$. Here, $A^{B}$ is a corresponding sub-matrix of $A$ and $b^{B}$ is the corresponding right-hand-side of the constraint. Let $\rho_{\min}(A^{B})$ denote the smallest singular value of $A^{B}$. Let $Act(D)$ denote the set of all active sets corresponding to vertices of $D$, i.e., $Act(D)=\{B:V^{B}\text{ is a vertex of }D\}$. Furthermore, define $\rho_{\min}(D):=\min\{\rho_{\min}(A^{B}):B\in Act(D)\}$. Note that $\rho_{\min}(D)>0$ since by definition, $B$ is a set of linearly independent active constraints. Let $\varepsilon_{0}=\min_{i}\{b^{i}-[a^{i}]^{T}x_{0}\}$, and $L_{A}=\max_{i}\|a^{i}\|$. With the notation in place, we can present the following lemma on the lower bound on the number of measurements to ensure safety of each iterate.

Lemma 1**.**

If $\beta\in\mathcal{E}_{k}(\bar{\delta})$ for $k=1,\ldots,t$ and $n_{t}=4C_{n}t(\ln t)^{2}$, with the constant parameter $C_{n}$ satisfying

[TABLE]

[TABLE]

then $x_{t}\in S_{t}(\bar{\delta}).$ Furthermore, the total number of measurements then satisfies $N_{t}=C_{n}t^{2}(\ln t)^{2}$.

We provide the full proof in Appendix C.

Proof sketch.

Let us give a brief intuition for the proof. First, from Fact 2 we see that in order to have $x_{t}\in S_{t}(\bar{\delta})$ we require $\min_{i}\varepsilon_{t}^{i}\geq\Omega\left(\frac{1}{\sqrt{N_{t}}}\right)$, where $\varepsilon_{t}^{i}$ is equal to the distance to the boundary corresponding to the estimated $i$-th constraint multiplied by $\|\hat{a}_{t}^{i}\|$. Second, if these estimates were fixed, then using step sizes $\gamma_{t}=\frac{1}{t+2}$ we could ensure that the convergence to any boundary $i$ would not be faster than $\varepsilon_{t}^{i}\geq\prod_{k=0}^{t}(1-\gamma_{k})\varepsilon_{0}^{i}=\frac{\varepsilon_{0}^{i}}{t+2}$ (see Figure 1). Hence, we require $N_{t}\geq\Omega\left(\frac{t^{2}}{\varepsilon_{0}^{2}}\right)$. However, since $\varepsilon_{t}^{i}$ are random and boundaries are fluctuating, we need $N_{t}$ to be square logarithmic times larger than the above estimate (as shown in the full proof). ∎

Remark. Note that the dependence on $\ln\ln T$ is very mild because this term grows extremely slowly, i.e, $\ln\ln 15\approx 1$ and $\ln\ln 2000\approx 2$.

Having established Lemma 1, we can present the safety guarantee of the SFW algorithm.

Theorem 1**.**

If $n_{t}=4C_{n}t(\ln t)^{2}$, where the constant parameter $C_{n}$ is defined in (10) then, the sequence of iterates $\{x_{t}\}_{t=0}^{T}$ of SFW is feasible with probability at least $1-\delta$, that is, $\mathbb{P}\{x_{t}\in D\text{ for all }0\leq t\leq T\}\geq 1-\delta.$

Proof.

Let $\mathcal{F}_{t}$ denote the event that all the estimated confidence sets $\mathcal{E}_{k}(\bar{\delta})$ up to step $t$ cover $\beta$, i.e., $\mathcal{F}_{t}=\{\beta\in\cap_{k=0}^{t}\mathcal{E}_{k}(\bar{\delta})\}.$ Furthermore, let $\mathcal{Q}_{t}$ denote the event that all the $x_{k}\in S_{k}(\bar{\delta})$ up to iteration $t$, i.e, $\mathcal{Q}_{t}=\{x_{k}\in S_{k}(\bar{\delta})\text{ for all }0\leq k\leq t\}.$ By Lemma 1 if $\mathcal{F}_{t}$ holds, then $x_{t}\in S_{t}(\bar{\delta}).$ Hence, it is easy to see that $\mathcal{F}_{t}$ implies $\mathcal{Q}_{t}$, i.e., $\mathbb{P}\{\mathcal{Q}_{t}|\mathcal{F}_{t}\}=1.$ Thus, using Fact 1 we derive

[TABLE]

Using Boole’s inequality we can bound the probability of $\mathcal{F}_{T}$ as follows

[TABLE]

This concludes the proof. ∎

5 CONVERGENCE

First, we show that the proposed algorithm achieves the optimal convergence rate for the Frank-Wolfe algorithm (see Lan, (2013)), with sufficiently high probability. Second, we discuss extensions to stochastic first-order and zeroth-order oracles of the objective function based on the results of Hazan and Luo, (2016).

5.1 Convergence rate.

Let us define the curvature constant $C_{f}$ of the function $f(x)$ with respect to the compact domain $D$ by

[TABLE]

It can be verified that $C_{f}\leq L\Gamma^{2}$, where $L$ is the Lipschitz constant of the gradient $\nabla f(x)$ and $\Gamma$ is the diameter of the set $D$ (see Section 2). Our main result is as follows.

Theorem 2**.**

*If $n_{t}=4C_{n}{(t+2)(\ln(t+2)})^{2}$ and $C_{n}$ is chosen according to the bound in Equation (10), then:

a) after $T$ steps of the SFW algorithm, the final point $x_{T}$ satisfies*

[TABLE]

*where $C^{\prime}=\frac{MC_{\bar{\delta}}}{\sqrt{C_{n}}}$, and $C_{\bar{\delta}}$ is defined in (11).

b) all the iterates $\{x_{t}\}_{t=1}^{T}$ are feasible with probability $1-\delta$, as required in (3).*

Corollary 1**.**

The SFW algorithm achieves an $\epsilon$-accurate solution with probability greater than $1-\delta$ after making $\tilde{O}(\frac{1}{\epsilon})$ linear optimization oracle calls and $\tilde{O}\left(\max\left\{\frac{d^{3}\ln\frac{1}{\bar{\delta}}}{\epsilon^{2}},\frac{d^{2}\ln^{2}\frac{1}{\bar{\delta}}}{\epsilon^{2}}\right\}\right)$ zeroth-order inexact constraint oracle calls.

Below, we provide the proof sketch for Theorem 2. The full proofs of Theorem 2 and Corollary 1 are provided in Appendix D.

Proof sketch.

Our proof is based on the extensive study of FW convergence provided by Jaggi, (2013), Freund and Grigas, (2016). Recall that $E_{t}$ is the accuracy with which an approximated DFS at iteration $t$ is solved. Similarly to (Freund and Grigas, (2016), Theorem 5.1) we can show that for $\gamma_{t}=O\left(\frac{1}{t}\right)$, we have

[TABLE]

where $\epsilon_{0}=f(x_{0})-f(x_{*})$. Hence, to prove the convergence rate of the SFW algorithm we need to show that the error in the DFS solution decreases with the rate $O\left(\frac{1}{t}\right)$. This fact is shown in Proposition 1 below.

Proposition 1**.**

If $\beta\in\mathcal{E}_{t}(\bar{\delta})$ and $N_{t}\geq\frac{C_{\bar{\delta}}^{2}}{(\Gamma_{0}+1)^{2}},$ then $E_{t}\leq\frac{MC_{\bar{\delta}}}{\sqrt{N_{t}}}.$ Since $\mathbb{P}\{\beta\in\mathcal{E}_{t}(\bar{\delta})\}\geq 1-\bar{\delta}$, we obtain $\mathbb{P}\left\{E_{t}\leq\frac{MC_{\bar{\delta}}}{\sqrt{N_{t}}}\right\}\geq 1-\bar{\delta}.$

We provide the proof in Appendix B. From the result above it directly follows that $E_{t}=O\left(\frac{1}{t\ln t}\right)$, hence $\sum_{k=0}^{t}E_{k}=O\left(\ln\ln t\right)$. Using this result, and the classical FW proof technique (Freund and Grigas, (2016)) we can conclude the result of Theorem 2. This concludes the proof sketch. ∎

We can see that under our choice of the number of measurements $n_{t}$ at each iteration, the total number of measurements is $O\left({d^{2}}t^{2}\ln t^{2}\right)$. It follows that the required number of measurements at each step grows almost linearly with the iteration number and quadratically with the dimension $d$. Note however that the number of iterations is independent of the dimension $d$. In contrast, the safe learning approach in (Sui et al.,, 2015) is based on gridding the decision space and hence, the dependence in $d$ is exponential. Hence, compared to previous safe learning approaches (Sui et al.,, 2015; Berkenkamp et al.,, 2016), our method scales better with dimension. Naturally, this scalability is due to the assumption of convexity of the cost function and the linearity of the constraints.

Finally, let us clarify some computational complexity aspects. After adding each new data point to $X$, the matrix inversion $(X^{T}X)^{-1}$ in Step 6 can be performed using one-rank updates (e.g., using formula $(A+vv^{T})^{-1}=A^{-1}-\frac{1}{(1+v^{T}A^{-1}v)}(A^{-1}vv^{T}A^{-1}))$. The cost of each such operation is $O(d^{2})$. This operation is to be made $N_{t}=O(d^{2}t^{2}(\ln t)^{2})$ times. The total computational complexity is thus $O(d^{2}N_{t})=O(d^{4}t^{2}(\ln t)^{2})$ from Step 6 and additionally $t$ LP oracle calls in Step 7.

5.2 Extension to stochastic oracle for the objective function.

Notice that the SFW algorithm requires $t=\tilde{O}\left(\frac{1}{\epsilon}\right)$ iterations and $N_{t}=\tilde{O}\left(\frac{d^{2}\ln\frac{1}{\delta}}{\epsilon^{2}}\right)$ measurements of constraints to obtain a required accuracy of $\epsilon$. General Stochastic Frank-Wolfe algorithm with stochastic objective but known linear constraints require $t=O(\frac{1}{\epsilon})$ iterations and in contrast, $t=O(\frac{1}{\epsilon^{3}})$ stochastic gradient measurements (Hazan and Luo, (2016), Table 2).333The projection-free scheme for stochastic optimization proposed in (Lan and Zhou,, 2016) achieves $N_{t}=O\left(1/\epsilon^{2}\right)$ measurements in total, but their method significantly differs from the original FW and the one used in the current paper. This difference in the number of measurements is due to the fact that in the absence of linearity of the objective function, the gradients of the objective function are changing in each iteration. Thus, $O(t^{2})$ measurements at each iteration are needed to guarantee correct variance reduction rate of the Frank-Wolfe method (see (12)). From the above observation, we can extend the SFW analysis to the case in which we have access to a stochastic first-order oracle of the objective function. In this case, a total of $O(t^{3})$ calls to the objective function oracle, and $O(t^{2})$ calls to the constraints oracle are sufficient to obtain the desired rate of decrease of $E_{t}$ in Proposition 1 and hence, the convergence rate in Theorem 2. Similarly, for the case of zeroth-order oracle $O(d^{2}t^{3})$ calls are needed to estimate the gradient of the objective. The noisy gradient does not influence the safety of the proposed algorithm. Thus, the safety results in Theorem 1 extend to the case with stochastic first-order or zeroth-order oracle of the objective.

6 EXPERIMENTS

We evaluate the performance of the proposed approach experimentally. In the first experiment, we consider the convergence rate of the algorithm as a function of the dimension. In the second experiment, we compare the SFW algorithm with a robust optimization based approach, which first learns the uncertain constraints and then finds the optimum with respect to the estimated constraints. We consider the convex smooth optimization problem:

[TABLE]

where $D=\left\{x\in\mathbb{R}^{d}|-1\leq x^{i}\leq 1,i=1,\ldots,d\right\}$ and $x^{\prime}=[2,0.5,\ldots,0.5]\in\mathbb{R}^{d}$ for varying dimension $d$. Then, the solution $x_{*}$ is a point on the boundary of the true constraint set above. We set the variance of the noise to $\sigma=0.01$ and use a constant exploration radius $\omega_{0}=0.01$. Furthermore, we set the confidence parameter $\delta=0.1$ and the total number of iterations to $T=15$.

Empirical constraint violation and convergence rate.

The first experiment evaluates the empirical convergence rate and constraint violation as a function of the dimension $d$. First, we evaluate the convergence rate assuming we can obtain the required lower bound on $C_{n}$ as per Theorem 2. We run the algorithm for dimensions $d=2,10,20$. In particular, the parameters of the problem required for obtaining the lower bound are derived based on knowledge of the constraints and the objective function, as well as the input parameters $\delta$, $T$ as follows: $\varepsilon_{0}=1,\bar{\delta}=0.0067,\phi_{\bar{\delta}}=3.43$, $\|b\|=2,\|a\|=1.$ It follows that a value of $C_{n}=d^{2}\cdot 24$ achieves the required number of measurements. The SFW proposed in Algorithm 1 is then run with the above choices of parameters $\bar{\delta}$ and $n_{t}$. For each dimension, we run the SFW algorithm 20 times, keeping all the parameters and the initial conditions the same. The difference in each experiment is due to the stochastic noise in the measurements. The average and standard deviation of the function values $f(x_{t})-f(x_{*})$ scaled by the initial condition error are shown in Figure 2. It can be seen that the dimension does not influence the convergence rate, rather, it influences only the number of measurements.

We also run the experiment assuming we cannot compute the lower bound $C_{n}$ precisely due to lack of problem data. In this case, at each iteration we first take $2dt$ measurements and further continuously take new measurements around $x_{t}$ until the safety set $S_{t+1}({\bar{\delta}})$ grows sufficiently to ensure $x_{t+1}$ becomes safe (see Fact 2). This safety indeed verifies the feasibility of iterates with high probability based on Fact 1. Let us refer to this as the adaptive variant of SFW. This adaptive approach is not only more practical due to lack of dependence on problem data, but also requires far fewer measurements in total since the bound on $n_{t}$ from the Theorem 1 is quite conservative. This is due to the fact that our theoretical bounds were derived using the worst-case estimate of $\|\Sigma_{t}^{1/2}\|$, when the measurements are taken always around the same point. However, $\|\Sigma_{t}^{1/2}\|$ can reduce much faster in practice. The convergence will also hold since the bound on $E_{t}\leq\frac{C_{\bar{\delta}}M}{\sqrt{N_{t}}}\leq\frac{C_{\bar{\delta}}M}{t+2}=O\left(\frac{1}{t}\right)$ required for the convergence rate is still satisfied. In Figure 2 caption we reported the required total number of measurements up to step $T$ of the adaptive variant by $N_{T}^{a}$. You can see that it has significantly reduced compared to the non-adaptive variant.

Comparison with an alternative robust optimization approach.

We compare the proposed SFW method with an alternative approach in which we first make enough measurements in the a priori safe region to estimate the safety set (see Definition (8)) with sufficiently high probability. Next, we run a first-order method, such as FW with respect to the nonlinear set $S({\bar{\delta}})$. Let us call this approach RO for robust optimization. To compare these two methods we set an a priori number of measurements for the alternative RO method equal to the total number of measurements $N_{t}$, of SFW algorithm corresponding to $\delta=0.1$ and $T=15$. After estimation of the safety set, we make $T=15$ iterations of the FW method with the constraint set ${S_{T}}({\bar{\delta}})$. Thus, the total number of measurement and the total number of optimization steps of the two methods are equal. During the run of the SFW, as per discussion in the above example, we reduce the number of measurements required to ensure safety at each iteration $t$, online. Figure 3(a),(b) shows the optimization trajectories of each method. The green round points along the trajectory correspond to the points where the constraints were measured. We also show the comparison of their convergence rates in Figure 3(c). As we can see, SFW algorithm performs better both in terms of estimates of the constraints and convergence rate. This difference in performance can be explained based on two observations. First, SFW moves measurements along the trajectory $\{x_{t}\}$, and this can lead to smaller variance of the estimates $\Sigma_{t}=\sigma^{2}\left(\bar{X}_{t}^{T}\bar{X}_{t}\right)^{-1}$. Hence, the measurements are more informative and the safety set $S_{T}({\bar{\delta}})$ is larger. Second, SFW algorithm is proven to converge to an $\epsilon$-optimal solution corresponding to the true constraints. The RO approach however, can at the very best converge to an optimum with respect to a safety set estimated in advance. From the computational perspective, at each iteration the proposed SFW method requires an LP oracle, whereas the alternative RO approach requires solving a second-order cone program. Hence, the SFW is more tractable.

7 CONCLUSION

We proposed a safe learning approach for convex costs and uncertain linear constraints. This method uses information along the optimization trajectory to decrease the objective value and to explore an unknown feasible set. Meanwhile, it ensures feasibility for each iteration with high probability. We provided an analysis of convergence rate of our algorithm, as well as of feasibility guarantees for its iterations. Our next steps are to generalize the results to nonlinear constraints and to provide performance guarantees in terms of regret.

8 ACKNOWLEDGEMENTS

We thank Johannes Kirschner and Michel Baes for their helpful comments.

Appendix A Proof of Fact 2

Proof.

Recall that the safety set $S_{t}(\bar{\delta})$ after iteration $t$ is defined by the following inequalities:

[TABLE]

Remember that $\bar{x}_{t}=\frac{X^{T}\mathbf{1}}{N}$ is an average of the measured points. Using the inversion formula for a block matrix, we obtain

[TABLE]

where

[TABLE]

Let us denote by $\phi_{\bar{\delta}}=\sigma\phi^{-1}(\bar{\delta}/m)$ and by $\varepsilon^{i}_{t}=\hat{b}_{t}^{i}-(\hat{a}_{t}^{i})^{T}x_{t}$. Then, the $i$-th inequality in (A) can be rewritten as follows:

[TABLE]

Substituting $x=x_{t}$ to the above and combining the inequalities together, we obtain that the condition $x_{t}\in S_{t}(\bar{\delta})$ is equivalent to

[TABLE]

∎

Appendix B DFS solution proof

Let us recall that for the polytope $D\in\mathbb{R}^{d}$, by an active set $B$ we denote a set of indices of $d$ linearly independent constraints active in a vertex $V^{B}\in\mathbb{R}^{d}$ of $D$, i.e., $V^{B}=[A^{B}]^{-1}b^{B}$. Here, $A^{B}$ is a corresponding sub-matrix of $A$ and $b^{B}$ is the corresponding right-hand-side of the constraint. The vertex estimate $\check{V}^{B}_{t}$ of a polytope is described by the system of linear equations $\hat{A}^{B}_{t}x=\hat{b}^{B}_{t}$.

Lemma 2**.**

If $\beta\in\mathcal{E}_{t}(\bar{\delta})$ and $N_{t}\geq\frac{C_{\bar{\delta}}^{2}}{(\Gamma_{0}+1)^{2}}$, where $C_{\bar{\delta}}$ is defined in (11), then for any vertex $V^{B}$ defined by the active set $B$ and its estimate $\check{V}^{B}_{t}$ we have that the estimation error is bounded by

[TABLE]

where $C_{\bar{\delta}}=\frac{2\phi_{\bar{\delta}}d(\Gamma_{0}+1)}{\rho_{\min}[A^{B}]}\sqrt{\frac{\Gamma_{0}^{2}+1}{\omega_{0}^{2}}+1}.$

Proof.

Since the LSE (Least Squares Estimation) is unbiased,

[TABLE]

Let us denote by $\zeta_{t}=\hat{b}_{t}^{B}-b^{B}$ the uncertainty in estimation of $b^{B}$, and $G_{t}=\hat{A}_{t}^{B}-A^{B}$ the uncertainty in estimation of $A^{B}$.

Our aim is to bound the error of the vertex estimation $\|\check{V}^{B}_{t}-V^{B}\|$. Recall that

[TABLE]

Note that for any matrices $A,B$ it holds that

[TABLE]

Therefore, we can modify the expression for the $\check{V}^{B}_{t}-V^{B}$ as follows

[TABLE]

The norm of the difference between the vertex $V^{B}$ of the set $D$ and its estimation can be bounded by

[TABLE]

To obtain the bounds on the terms (a),(b),(c),(d), let us first obtain the bounds on $\|G_{t}\|$ and $\|\zeta_{t}\|$.

Assume that for each $i=1,\ldots,m$ $\beta^{i}\in\mathcal{E}^{i}_{t}(\bar{\delta})$, where

[TABLE]

i.e., that for any active set $B$ describing the vertex $V^{B}$ we have $\beta^{B}\in\mathcal{E}^{B}_{t}(\bar{\delta})$. Consequently, $\|\hat{a}^{i}_{t}-a^{i}\|^{2}+|\hat{b}^{i}-b^{i}|^{2}\leq\phi^{-1}(\bar{\delta})\|\Sigma_{t}^{1/2}\|\leavevmode\nobreak\ \forall i\in B.$ Then, for each row of $G_{t}$ we have $\|\hat{a}^{i}_{t}-a^{i}\|\leq\phi^{-1}(\bar{\delta})\|\Sigma_{t}^{1/2}\|$, and for each element of $\zeta_{t}$ we have $|\hat{b}^{i}_{t}-b^{i}|\leq\phi^{-1}(\bar{\delta})\|\Sigma_{t}^{1/2}\|.$ Hence, for $\|G_{t}\|$ we obtain

[TABLE]

Similarly, we obtain a bound on $\|\zeta_{t}\|$:

[TABLE]

For the LSE covariance matrix norm $\|\Sigma_{t}^{1/2}\|$ we have

[TABLE]

where $R_{t}$ was defined in (A).

Note that we make measurements as it is described in Step 4 of the SFW algorithm, i.e., we make measurements at all coordinate directions within small step size $\omega_{0}$ from points generated by the method. Then, each new $2d$ measurements result in addition of a matrix $\sum_{j=N_{t}+1}^{N_{t}+2d}(x_{(j)}-\bar{x})(x_{(j)}-\bar{x})^{T}\succeq\omega_{0}^{2}I$ to the matrix $R_{t}^{-1}=\sum_{j=1}^{N_{t}}(x_{(j)}-\bar{x}_{t})(x_{(j)}-\bar{x}_{t})^{T}$. Hence, $R_{t}^{-1}\succeq\frac{N_{t}\omega_{0}^{2}}{2d}I$. Hence, the minimal eigenvalue of the covariance matrix $R_{t}^{-1}$ is bounded from below by the value $\lambda_{\min}(R_{t}^{-1})\geq\frac{N_{t}\omega_{0}^{2}}{d}.$ Thus, we obtain the following bound on the norm of $R_{t}$:

[TABLE]

Note that $\bar{x}_{t}\in D$, hence $\|\bar{x}_{t}\|\leq\Gamma_{0}$. It follows that

[TABLE]

In order to bound terms (a),(b),(c),(d) in inequality (B), let us also bound the norm of the matrix $[A^{B}]^{-1}$:

[TABLE]

Then, combining inequalities (16),(17),(B),(20), we bound terms (a) and (b) as follows:

[TABLE]

[TABLE]

where $U$ is defined by

[TABLE]

Further, let us bound term (d). For $N_{t}\geq 4U^{2}$ it holds that

[TABLE]

As such, for $N_{t}\geq 4U^{2}$ we have

[TABLE]

Hence

[TABLE]

Finally, term (c) can be bounded as follows:

[TABLE]

Combining these all together, we obtain

[TABLE]

where

[TABLE]

Since $N_{t}=C_{n}t^{2}(\ln t^{2})\geq C_{n}$, the above bound holds under the proper choice of the constant

[TABLE]

∎

Proposition 1

Proof.

Let us bound the difference between the solution of the estimated DFS and the solution of the true DFS. Estimated DFS is a linear program defined by

[TABLE]

where $c_{t}=\nabla f(x_{t})$. Any solution of such a linear program is a vertex (or convex hull of vertices) of the polytope $\hat{A}_{t}x\leq\hat{b}_{t}$. Let $\hat{V}^{2}_{t}$ be the estimated DFS solution vertex $\hat{V}^{2}_{t}=\hat{s}_{t}=\arg\min_{s\in\hat{D}_{t}}\langle c_{t},s\rangle.$ And correspondingly, let $V^{1}_{t}$ be the true DFS solution $V^{1}_{t}=s_{t}=\arg\min_{s\in D}\langle c_{t},s\rangle.$

Let us define by $\pi_{\hat{D}_{t}}V^{1}_{t}$ the projection of $V^{1}_{t}$ onto $\hat{D}_{t}$: $\bar{V}^{1}_{t}=Proj_{\hat{D}_{t}}V^{1}_{t},$ and correspondingly we define $\tilde{V}^{2}_{t}$ as $\tilde{V}^{2}_{t}=Proj_{D}\hat{V}^{2}_{t}$. Recall that the estimate $\check{V}^{B}$ of any vertex $V^{B}=[A^{B}]^{-1}b^{B}$ of the polytope $D$ is described by the system of linear equations $\hat{A}^{B}_{t}x=\hat{b}^{B}_{t}$. Since the estimates $\check{V}^{B}$ denote intersections of the hyper-planes $\langle\hat{a}^{j},s\rangle=\hat{b}^{j}\leavevmode\nobreak\ \forall j\in B$ for some particular subset of indices $B$, the polytope $\hat{D}_{t}$ lies inside the convex hull of the estimates $\check{V}^{B}_{t}.$ Hence, any point $s\in\hat{D}_{t}$ cannot be further from $D$ than the estimates of all the vertices $\check{V}^{i}$ from $D$. By Lemma 2, if $N_{t}\geq\frac{C_{\bar{\delta}}^{2}}{(\Gamma_{0}+1)^{2}}$ with $C_{\bar{\delta}}$ defined in (11) and $\beta\in\mathcal{E}_{t}(\bar{\delta})$, then for any vertex $V^{B}$ of $D$ we have $\|V^{B}-\check{V}^{B}_{t}\|\leq\frac{C_{\bar{\delta}}}{\sqrt{N_{t}}}$, thus we obtain that the distance from any point $s\in\hat{D}_{t}$ to $D$ is less than $\frac{C_{\bar{\delta}}}{\sqrt{N_{t}}}$. I.e., we have $\|\bar{V}^{1}_{t}-V^{1}_{t}\|\leq\frac{C_{\bar{\delta}}}{\sqrt{N_{t}}}.$

Similarly, we show that the distance from any point $s\in D$ to the set $\hat{D}_{t}$ is upper bounded by $\frac{C_{\bar{\delta}}}{\sqrt{N_{t}}}$. Again, we can see that $D$ is bounded by the convex hull of $V^{\hat{B}}$, where each $V^{\hat{B}}=[A^{\hat{B}}]^{-1}b^{\hat{B}}$ corresponds to a vertex $\check{V}^{\hat{B}}_{t}=[\hat{A}_{t}^{\hat{B}}]^{-1}\hat{b}_{t}^{\hat{B}}$ of $\hat{D}_{t}$. Hence , the distance from any point $s\in D$ to the set $\hat{D}_{t}$ is upper bounded by $\max_{\hat{B}}\|V^{\hat{B}}-\check{V}^{\hat{B}}_{t}\|.$ Then, by Lemma 2 we have $\|\tilde{V}^{2}_{t}-\hat{V}^{2}_{t}\|\leq\frac{C_{\bar{\delta}}}{\sqrt{N_{t}}}$.

Note that $\bar{V}^{1}_{t}\in\hat{D}_{t}$, $\tilde{V}^{2}_{t}\in D$. From the definitions of $\hat{V}^{2}_{t},V^{1}_{t}$ above it follows that

[TABLE]

Hence, we have

[TABLE]

Thus, we obtain that if $\beta\in\mathcal{E}_{t}(\bar{\delta})$, then

[TABLE]

Note that $\|c_{t}\|\leq M$, where $M$ is the Lipschitz constant of the objective. Thus, if $\beta\in\mathcal{E}_{t}(\bar{\delta})$, then $E_{t}\leq\frac{C_{\bar{\delta}}M}{\sqrt{N_{t}}}$, i.e.,

[TABLE]

∎

Appendix C Proof of Lemma 1

First, we provide some preliminary lemmas for the proof of Lemma 1.

Let us denote by ${\breve{x}_{t}}={x_{t}}-\bar{x}_{t}$, $\Delta^{k}_{t}=\hat{s}_{{t}}-{x_{k}}$, and recall that $\varepsilon^{i}_{t}=\hat{b}_{t}^{i}-[\hat{a}_{t}^{i}]^{T}{x_{t}}$. By $\hat{V}_{t-1}$ we call the solution of the estimated DFS at the step $t-1$, by $\tilde{V}_{t-1}=Proj_{D}\hat{V}_{t-1}$, and by $\bar{V}_{t,t-1}=Proj_{\hat{D}_{t}}V_{t-1}$. By $B_{t}$ we denote the active set corresponding to $\hat{V}_{t}$ (see Lemma 1 for definition) and by $B_{t-1}$ the active set corresponding to $\hat{V}_{t-1}$

Lemma 3**.**

If $\beta\in\mathcal{E}_{t}(\bar{\delta})\cap\mathcal{E}_{t-1}(\bar{\delta})$ holds, then we have

[TABLE]

Proof.

[TABLE]

For the point $s_{t}$ we have $\langle\hat{a}^{j}_{t},s_{t}\rangle=0$ and for any point $s\in\hat{D}_{t}$ we have $\langle\hat{a}_{t}^{j},s\rangle\geq 0$. Note that $\hat{V}_{t,t-1}\in\hat{D}_{t}$ and that $\|\hat{V}_{t-1}-\hat{V}_{t,t-1}\|\leq\|\hat{V}_{t-1}-V_{t-1}\|+\|V_{t-1}-V_{t,t-1}\|.$

[TABLE]

Using the result of Lemma 2 we can obtain that if $\beta\in\mathcal{E}_{t}(\bar{\delta})\cap\mathcal{E}_{t+1}(\bar{\delta})$ and $N_{t}\geq\frac{C_{\bar{\delta}}^{2}}{(\Gamma_{0}+1)^{2}}$ with $C_{\bar{\delta}}$ defined in (11), and applying the same arguments as in the proof of Proposition 1, we obtain

[TABLE]

Then, combining (C) with (C), we have

[TABLE]

∎

Lemma 3 above is an induction step in the proof of Lemma 4. Lemma 4 below bounds the fastest rate of decreasing the distance to the boundaries of $D$ for the SFW algorithm. Recall that $\mathcal{F}_{t}=\{\beta\in\cap_{k=0}^{t}\mathcal{E}_{k}(\bar{\delta})\}.$

Lemma 4**.**

If $\mathcal{F}_{t}$ holds, then we have

[TABLE]

Proof.

By induction, from Lemma 3 we have

[TABLE]

Note that $1-\gamma_{k}=\frac{k+1}{k+2}$, and

[TABLE]

[TABLE]

Recall that $N_{t}=C_{n}t^{2}(\ln t)^{2}$, hence we have

[TABLE]

∎

With Lemmas 3 and 4 in place, we are ready to prove Lemma 1.

C.1 Proof of Lemma 1

Proof.

From Fact 2, the condition $x_{t}\in S_{t}(\bar{\delta})$ is equal to

[TABLE]

From the bound on $\|R_{t}\|$ given in (18) and knowing that $\Gamma$ is a diameter of the set $D$, we have

[TABLE]

From Lemma 4 and recalling that $\varepsilon_{t}^{i}=\min_{i}\langle\hat{a}^{i}_{t},\Delta_{t}^{t}\rangle$, we have

[TABLE]

Hence, we can guarantee that $x_{t}\in S_{t}(\bar{\delta})$ if

[TABLE]

We denote by $L_{A}=\max_{i}\|a_{i}\|$. Let us derive how far are $\min_{i}\langle\hat{a}_{t}^{i},{{\Delta}_{t}^{0}}\rangle$ from $\varepsilon_{0}$ and $\max_{i}\|\hat{a}_{t}^{i}\|$ from $L_{A}$. These are needed for obtaining a bound on the denominator above. Let us define by $\Delta_{0}=s_{t}-x_{0}$, where $s_{t}$ is the true vertex of $\hat{s}_{t}$ corresponding to $\min_{i}\langle\hat{a}_{t}^{i},{{\Delta}_{t}^{0}}\rangle$. Also recall that then $\min_{i}[\varepsilon_{0}]\leq\langle a^{i},\Delta_{0}\rangle$. If $C_{n}\geq\frac{C_{\bar{\delta}}^{2}}{(\Gamma_{0}+1)^{2}}$, then with probability greater than $1-\bar{\delta}$ we have $\|{\Delta}_{t}^{0}-\Delta_{0}\|\leq\frac{C_{\bar{\delta}}}{\sqrt{N_{t}}}$(using Lemma 2). We also can bound the difference $\|\hat{a}^{i}_{t}-a^{i}\|$ by $\|\hat{a}^{i}_{t}-a^{i}\|\leq\phi^{-1}(\bar{\delta})\|\Sigma^{1/2}\|\leq\frac{C_{\bar{\delta}}}{\sqrt{N_{t}}}\frac{1}{\sqrt{d}\rho_{\min}(D)(\Gamma_{0}+1)}.$ The second inequality follows from (B) and definition of $C_{\bar{\delta}}$ (11).

Combining above inequalities together with the bound (26) on $N_{t}$ we can conclude the following. If

[TABLE]

then we can guarantee that $x_{t}\in S_{t}(\bar{\delta})$ by requiring

[TABLE]

Since $n_{t}=C_{n}(t+1)(\ln(t+2))^{2}$ and $N_{t}=\sum_{k=0}^{t}n_{k}$, we obtain that

[TABLE]

Hence, $C_{n}\geq\frac{\phi_{\bar{\delta}}^{2}\left(1+\frac{d\Gamma_{0}^{2}}{\omega_{0}^{2}}\right)}{(\ln(t+2))^{2}\min_{i}[\varepsilon^{i}_{0}]^{2}}$ is enough to ensure that $x_{t}\in S_{t}(\bar{\delta})$. Note that $C_{\bar{\delta}}\geq\phi_{\bar{\delta}}^{2}\left(1+\frac{d\Gamma^{2}}{\omega_{0}^{2}}\right)$. Thus, under the proper choice of constant parameter $C_{n}$, namely,

[TABLE]

we conclude that $x_{t}\in S_{t}(\bar{\delta}).$ ∎

Remark

Note that if we use a step size as in classical FW, $\gamma_{t}=\frac{2}{t+2}$, or in more general form $\gamma_{t}=\frac{l}{t+l}$ then we obtain that the distance to the boundaries $\min_{i}[\varepsilon_{t}^{i}]$ will decrease with the rate upper bounded by $\prod_{k=0}^{t}(1-\gamma_{k})=\frac{l!}{t\cdot\ldots\cdot(t+l)}=O(\frac{1}{t^{l}})$ instead of (25) and this bound can be achieved e.g. in the case if the algorithm always moves in the same direction towards the boundary. This implies that in order to keep the convergence rate as in the original FW while satisfying $x_{t}\in S_{t}(\bar{\delta})$, due to Fact 2 we have to reduce the uncertainty of the boundaries faster, i.e., we need to take more measurements at each iteration.

Appendix D Theorem 2

Proof.

Let $\mu_{t}(\bar{\delta})$ denote a constant such that $\bar{E}_{t}(\bar{\delta})=\frac{1}{2}\mu_{t}(\bar{\delta})\gamma_{t}C_{f}$. Then with probability $1-\bar{\delta}$ we have

[TABLE]

For the proof we refer to the following result from Jaggi, (2013). This result holds in our setting since we use the same notions as in Jaggi, (2013) of $g_{t}$ and $s_{t}$ defined in (7).

Lemma 5**.**

(Lemma 5 Jaggi, (2013)) For a step $x_{t+1}=x_{t}+\gamma(\hat{s}-x_{t})$ with an arbitrary step-size $\gamma\in[0,1]$, it holds that

[TABLE]

if $\hat{s}$ is an approximate linear minimizer, i.e.

[TABLE]

The step-size of the SFW algorithm is equal to $\gamma_{t}=\frac{1}{t+2}$. Let us define $h_{t}$ as follows

[TABLE]

Then we obtain that

[TABLE]

If we continue in the same manner, we obtain

[TABLE]

Recall that $\bar{E}_{t}(\bar{\delta})$ denotes the upper bound on $E_{t}$ with the confidence level $1-\bar{\delta}$.

Due to Proposition 1, we have

[TABLE]

Hence, we obtain that

[TABLE]

Therefore, we obtain

[TABLE]

where $C^{\prime}=\frac{MC_{\bar{\delta}}}{\sqrt{C_{n}}}.$ ∎

Corollary 1

Proof.

Recall that

[TABLE]

Hence,

[TABLE]

Recall that the total number of measurements $N_{t}$ satisfies

[TABLE]

Hence, we conclude

[TABLE]

∎