A Data Efficient and Feasible Level Set Method for Stochastic Convex Optimization with Expectation Constraints
Qihang Lin, Selvaprabu Nadarajah, Negar Soheili, Tianbao Yang
TL;DR
This paper introduces a stochastic feasible level set method (SFLS) for solving large-scale stochastic convex optimization problems with expectation constraints, emphasizing early feasibility and low data complexity.
Contribution
The paper develops a novel SFLS algorithm that maintains feasibility at each step and improves data efficiency over existing methods for stochastic convex optimization with constraints.
Findings
SFLS achieves high-probability feasibility at each iteration.
SFLS demonstrates lower data complexity than existing methods.
Numerical results show faster feasible solution attainment with small optimality gaps.
Abstract
Stochastic convex optimization problems with expectation constraints (SOECs) are encountered in statistics and machine learning, business, and engineering. In data-rich environments, the SOEC objective and constraints contain expectations defined with respect to large datasets. Therefore, efficient algorithms for solving such SOECs need to limit the fraction of data points that they use, which we refer to as algorithmic data complexity. Recent stochastic first order methods exhibit low data complexity when handling SOECs but guarantee near-feasibility and near-optimality only at convergence. These methods may thus return highly infeasible solutions when heuristically terminated, as is often the case, due to theoretical convergence criteria being highly conservative. This issue limits the use of first order methods in several applications where the SOEC constraints encode implementation…
| Dataset | Number of classes | Number of instances | Number of features |
|---|---|---|---|
| connect-4 | 3 | 67557 | 126 |
| covtype | 7 | 581012 | 54 |
| news20 | 20 | 15935 | 62061 |
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
\OneAndAHalfSpacedXI\TheoremsNumberedThrough\ECRepeatTheorems\EquationsNumberedThrough\MANUSCRIPTNO
MS-17-01263.R2
\RUNAUTHOR
\RUNTITLE
Stochastic Level-Set Method
\TITLE
A Data Efficient and Feasible Level Set Method for Stochastic Convex Optimization with Expectation Constraints
\ARTICLEAUTHORS\AUTHOR
Qihang Lin \AFFTippie College of Business, The University of Iowa, 21 East Market Street, Iowa City, IA 52242, USA, \[email protected] \AUTHORSelvaprabu Nadarajah \AFFCollege of Business Administration, University of Illinois at Chicago, 601 South Morgan Street, Chicago, Illinois, 60607, USA, \[email protected] \AUTHORNegar Soheili \AFFCollege of Business Administration, University of Illinois at Chicago, 601 South Morgan Street, Chicago, Illinois, 60607, USA, \[email protected] \AUTHORTianbao Yang \AFFCDepartment of Computer Science, The University of Iowa, 21 East Market Street, Iowa City, IA 52242, USA, \[email protected]
\ABSTRACT
Stochastic convex optimization problems with expectation constraints (SOECs) are encountered in statistics and machine learning, business, and engineering. In data-rich environments, the SOEC objective and constraints contain expectations defined with respect to large datasets. Therefore, efficient algorithms for solving such SOECs need to limit the fraction of data points that they use, which we refer to as algorithmic data complexity. Recent stochastic first order methods exhibit low data complexity when handling SOECs but guarantee near-feasibility and near-optimality only at convergence. These methods may thus return highly infeasible solutions when heuristically terminated, as is often the case, due to theoretical convergence criteria being highly conservative. This issue limits the use of first order methods in several applications where the SOEC constraints encode implementation requirements. We design a stochastic feasible level set method (SFLS) for SOECs that has low data complexity and emphasizes feasibility before convergence. Specifically, our level-set method solves a root-finding problem by calling a novel first order oracle that computes a stochastic upper bound on the level-set function by extending mirror descent and online validation techniques. We establish that SFLS maintains a high-probability feasible solution at each root-finding iteration and exhibits favorable iteration complexity compared to state-of-the-art deterministic feasible level set and stochastic subgradient methods. Numerical experiments on three diverse applications validate the low data complexity of SFLS relative to the former approach and highlight how SFLS finds feasible solutions with small optimality gaps significantly faster than the latter method.
1 Introduction
Consider the stochastic optimization problem with expectation constraints (SOEC)
[TABLE]
where is a nonempty closed convex set, , , is a random vector whose probability distribution is supported on set , and is continuous and convex in for each realization of for . Given , a solution is called -feasible if . A solution is referred to as -optimal if . Alternatively, optimality can be measured relative to an initial feasible solution . In this case, we say is relative -optimal with respect to if .
Problem (1) is pervasive in stochastic optimization and appears as a central challenge in semi-supervised learning (Chapelle et al. 2009), shape-restricted regression (Seijo et al. 2011, Sen and Meyer 2017, Lim 2014, Cotter et al. 2016, Fard et al. 2016), Neyman-Pearson classification (Tong et al. 2016, Rigollet and Tong 2011, Tong 2013, Zhao et al. 2015), approximate linear programming and related relaxations (de Farias and Van Roy 2003, Adelman and Mersereau 2013, Nadarajah et al. 2015), portfolio selection (Markowitz 1952, Abdelaziz et al. 2007), risk management (Rockafellar and Uryasev 2000), supply chain design (Azaron et al. 2008), and multi-objective stochastic programming (Marler and Arora 2004, Abdelaziz 2012, Mahdavi et al. 2013, Barba-Gonzaléz et al. 2017). In this paper, we focus on overcoming the challenges of applying existing methods for solving SOECs in settings that are both data rich and where expectation constraints capture requirements that cannot be violated during real-world implementation.
In data-rich environments, each expectation appearing in (1) is defined by a data set containing a large number of data points (possibly infinite). The number of data points used when solving SOEC is an important computational bottleneck, which we refer to as the data complexity of an algorithm. Traditional approaches for solving SOECs can lead to large data complexity. For instance, consider the popular strategy of replacing each expectation in (1) by a sample average approximation (SAA; Shapiro 2013, Oliveira and Thompson 2017) and solving the resulting model using a deterministic iterative method (see, e.g., Nesterov 2004, Soheili and Pena 2012, and references therein). If the number of samples used to construct SAAs is small, the solution from the deterministic approximation may be highly infeasible to the original SOEC, in addition to being suboptimal (Shapiro 2013, Oliveira and Thompson 2017). Instead, if a large number of samples are used in each SAA, then the data complexity becomes large because the gradient or objective function evaluation at each iteration requires using a significant portion of each of the data sets.
In contrast, stochastic first order methods for tackling stochastic optimization problems have low per-iteration cost and data complexity and thus play a central role in machine learning packages such as TensorFlow and PyTorch (Robbins and Monro 1951, Nemirovski et al. 2009, Lan 2012, Ghadimi and Lan 2012, 2013, Chen et al. 2012, Lan et al. 2012, Schmidt et al. 2013, Shalev-Shwartz et al. 2017, Lan and Zhou 2015, Lin et al. 2014, Duchi and Singer 2009, Xiao and Zhang 2014, Xiao 2010, Hazan and Kale 2011, Bach and Moulines 2013, Allen-Zhu 2017, Goldfarb et al. 2017). These methods update solutions using stochastic gradients that can be computed using a small number of sampled data points. Stochastic first order methods typically ensure feasibility via projections onto a convex set at each iteration, where the convex set is assumed to be simple (e.g. a box or ball) for computational tractability. This assumption limits the applicability of first order methods for solving SOECs with general non-linear constraints. Recently, Lan and Zhou (2016) and Yu et al. (2017) developed stochastic subgradient (SSG) methods devoid of projections for solving (1) with single () and multiple constraints (), respectively. The SSG methods in these papers guarantee an -optimal and -feasible solution only at convergence.
In practice, SSG methods are terminated before their conservative theoretical conditions are met. Premature termination may lead to highly infeasible and sub- or super- optimal solutions. While some deviation from optimality is likely acceptable, a highly infeasible solution may not be implementable. Such situations arise in several data science applications in machine learning, as well as, across business (e.g., operations and finance) and engineering domains. We elaborate on the practical need for feasibility in a few cases below.
- •
Fairness constraints: Enforcing fairness criteria when learning classifiers across multiple classes (e.g., male and female) has become important in machine learning (Goh et al. 2016). This learning problem can be cast as an SOEC where fairness is modeled via expectation constraints. Constraint violations lead to classifiers that are biased towards one or more classes.
- •
Risk constraints: Planning problems in supply chain management and portfolio optimization often include bounds on the Conditional Value at Risk (CVaR), which can be cast as expectation constraints (Fábián 2008, Chen et al. 2010). Such constraints also arise when modeling distributionally robust versions of chance constraints (Wiesemann et al. 2014) and when limiting misclassification risk (i.e., misclassification rates) in multi-class Neyman Pearson classification (Weston and Watkins 1998, Crammer and Singer 2002). The aforementioned problems can be formulated as SOECs. Solutions violating risk constraints will likely fail stress tests that are performed before implementation.
- •
Bounding property: Approximate linear programs (ALPs) are well-known models for approximating the value function of high-dimensional Markov decision processes (Schweitzer and Seidmann 1985, de Farias and Van Roy 2003), and in particular, are SOECs. A solution satisfying the ALP constraints provides an optimistic bound on the optimal policy value, which is useful to evaluate the suboptimality of heuristic policies. Infeasibility in an ALP setting thus voids this desirable bounding property.
Motivated by the importance of feasibility and the status quo of stochastic first order methods, we design an approach for solving SOECs that has low data complexity and provides high probability feasible solutions before convergence. As a first step, we cast SOEC as a root-finding problem involving a min-max level set function, which is challenging to solve because it is non-smooth and includes high-dimensional expectations in the SOEC objective and constraints. To solve this reformulation, we develop a stochastic feasible level-set method (SFLS) for root finding that requires evaluating a “good” upper bound (we will make this notion of goodness precise in later sections) on the challenging level set function at each iteration. We show that employing the mirror descent method (Nemirovski et al. 2009) for computing such an upper bound requires approximating expectations in SOEC using SAAs at each iteration, which as already discussed above, leads to high data complexity. To overcome this issue, we introduce an SSG method to upper bound the level-set function by combining mirror-decent and online validation techniques, and in particular, extending the latter technique, originally proposed for minimization problems (Lan et al. 2012), to handle saddle point formulations. This method only requires stochastic values and gradients of the objective and constraint functions, respectively, which can be constructed at low cost using a small number of samples of in (1), that is, it has low data complexity. Calls to our SSG method return high-probability feasible solutions, which allows it to maintain an implementable solution at each root-finding iteration.
We analyze the iteration complexity of SFLS to find a feasible solution path (i.e., sequence of feasible solutions) that becomes relative -optimal with high probability. It is encouraging that the dependence of this complexity on is , which is comparable to the method by Yu et al. (2017) (labeled YNW111We abbreviate this method by YNW using the first letters of the last names of the authors.) that also finds an -optimal solution but only guarantees -feasibility at convergence. In other words, the intermediate solutions generated by YNW are not necessarily feasible. There is indeed a cost for ensuring feasibility in SFLS, which appears in the form of its iteration complexity depending on a condition measure. Such condition measures do not influence the complexity of YNW.
For deterministic constrained convex optimization problems, the level-set method (DFLS) of Lin et al. (2018b) also guarantees a feasible solution path with its iteration complexity depending on a condition measure. In principle, these DFLS based approaches can be applied to solve SOECs by viewing them as deterministic problems. This perspective is restrictive because it entails computing expectations in for exactly or replacing them by SAAs. In either case, the data complexity of DFLS will be high for reasons analogous to the ones already discussed above related to the use of SAAs. Therefore, a fully stochastic approach is required to achieve low data complexity when solving SOECs. Lin et al. (2018a) extend DFLS using variance-reduced sampling, which requires the functions have a finite-sum structure with each summand taking a specific form.222In particular, Lin et al. (2018a) require each summand has the form of . Unfortunately, as a result, their method cannot be applied to SOECs with generic expectation while our method does not have such limitation and assumes little structure on the problems. We are not aware of prior efforts to develop a fully stochastic versions of level set methods for SOECs – SFLS in this paper fills this gap.
To assess the performance of SFLS, we provide implementation guidelines with supporting theory and numerically evaluate SFLS on three applications: (i) approximate linear programming for Markov decision processes, (ii) Neyman-Pearson multi-class classification with risk constraints, and (iii) learning a classifier with fairness constraints. Feasibility plays a key role in each of these applications for reasons mentioned earlier in the introduction. Approximate linear programs in the first application are known special cases of SOECs. For the latter two applications, we propose formulations that are SOECs. As algorithmic benchmarks, we consider YNW and DFLS. We find that SFLS delivers feasible solutions quicker than YNW and in several cases also leads to smaller optimality gaps. Moreover, when YNW computes infeasible solutions it is challenging to interpret its objective value since it can be superoptimal, an issue that does not arise with SFLS. Both SFLS and DFLS maintain feasible solution paths (with outer iterates) but SFLS produces feasible solutions with much smaller optimality gaps due to its lower data complexity. In other words, DFLS requires significantly more data passes to reduce the suboptimality of its solutions and will thus not be practical for solving SOECs based on large data sets. Our findings underscore two important algorithmic insights: (i) feasible SOEC solutions can be computed well before theoretical convergence criteria are satisfied but doing this hinges on methods being able to emphasize feasibility; and (ii) ensuring that these early feasible solutions have small optimality gaps requires approaches with low data complexity. Both these properties are true for SFLS, while only the first and second properties, respectively, hold for DFLS and YNW.
This paper is organized as follows. In §2, we introduce SFLS, analyze its oracle complexity, and present a saddle-point reformulation of an SOEC. In §3, we discuss how the well-known stochastic mirror descent algorithm provides an idealized stochastic oracle for SFLS and highlight issues that complicate its use. In §4, we propose and analyze a new stochastic oracle to overcome these issues. In §5, we analyze SFLS combined with this oracle and provide implementation guidelines. In §6, we perform a computational study to understand the performance of SFLS across three applications relative to two benchmark methods. We conclude in §7.
2 Stochastic Feasible Level-set Method
Level-set methods tackle a constrained convex optimization problem by transforming it into a one-dimensional root-finding problem that is a function of a scalar level parameter (Lemaréchal et al. 1995, Nesterov 2004). We develop in this section a stochastic and feasible level set method that adds to this framework. We make the following standard assumption throughout the paper, which ensures that a strictly feasible and sub-optimal solution exists. {assumption}[Strict Feasibility] There exists a strictly feasible solution such that
and .
The root-finding reformulation of (1) relies on the level-set function
[TABLE]
where is a level parameter and
[TABLE]
Note that the expectation constraints of SOEC are now in the objective function of (2). For a given , if then is a feasible solution to (1). Formulations (1) and (2) are further linked by known properties of , which are summarized in the following lemma (based on lemmas 2.3.4 and 2.3.6 in Nesterov 2004 and Lemma 1 in Lin et al. 2018b).
Lemma 2.1
It holds that
- (a)
* is non-increasing and convex in ;*
- (b)
;
- (c)
, if and , if .
Part (a) of Lemma 2.1 highlights that is non-increasing and convex. Moreover, its part (b) implies that is the unique root of . Therefore, one can use a root finding procedure to generate both a sequence of level parameters that converges to and an associated vector at each iteration . Computationally, when a level parameter is found, the solution provides an “approximate” solution to (1). From the perspective of feasibility, it is important whether we have or . To elaborate, if , then by Lemma 2.1(c) and the corresponding solution need not be feasible to (1). On the other hand, if , we have from Lemma 2.1(c) and the vector is indeed a feasible solution. A root finding scheme that ensures at each iteration will thus return a sequence of feasible solutions , that is a feasible solution path, where is such that for a given and, in addition, we have from . These inequalities imply that . Thus, is an -optimal and feasible solution to (1) and it follows that solving SOEC can be cast as a root-finding problem involving .
Applying a root-finding algorithm to solve requires the exact computation of at each iteration, which is difficult due to the nontrivial stochastic optimization in (2). Hence, we consider an inexact root-finding method, henceforth stochastic feasible level set method (SFLS), extending what is done in Lin et al. (2018b) and Aravkin et al. (2019) in a deterministic setting. Level set methods require an oracle to compute an approximation of . This approximation is used to update . A key element that we develop as part of SFLS is the notion of a stochastic oracle, which we introduce next.
Definition 2.2** **(Stochastic Oracle)
Given , , and , a stochastic oracle returns a value and a vector that satisfy the inequalities and with a probability of at least .
Lemma 2.3 clarifies the importance of the conditions underpinning the above definition to ensure a feasible solution to (1).
Lemma 2.3
Given , , , and , the vector returned by a stochastic oracle defines a feasible solution to (1) with probability of at least .
This lemma states that a stochastic oracle can recover a high probability feasible solution provided the optimality tolerance is less than .
Algorithm 1 formalizes the steps of SFLS to find an approximate root to . Its inputs include a stochastic oracle ; an initial level parameter value , which exists because we can set by Assumption 2; optimality and error tolerances and , respectively; a probability ; and a parameter that defines a step length as . SFLS begins from the level set defined by . At each iteration it executes lines 3 though 9. In line 3, SFLS computes a probability that is used in the stochastic oracle call of line 4 to obtain an approximation and a high probability feasible solution . The probability decreases with the iteration count , that is, the probabilistic guarantee required of the stochastic oracle becomes more stringent to ensure the entire solution path is feasible with probability of at least . Lines 5-7 model the termination condition, which involves checking whether the approximation is greater than or equal to . If this condition holds, then the algorithm halts and returns the incumbent solution . Otherwise, is updated to in line 8 using and . Line 9 increments the iteration counter. While SFLS belongs to the family of level set approaches, it differs from known deterministic level set methods (see, e.g., Lin et al. 2018b and Aravkin et al. 2019) in its update step, termination criterion, and stochastic oracle.
We define the notion of an input tuple to ease the exposition of theoretical statements in the rest of the paper.
Definition 2.4** **(Input tuple)
A tuple containing a subset of the elements , and is an input tuple if its respective components satisfy , , , , , , and , where is a constant that is formally defined in (11).
Theorem 2.5 provides the maximum number of calls to the stochastic oracle by Algorithm 1 to obtain a feasible and relative -optimal solution, which depends on a condition measure of SOEC (1) defined as
[TABLE]
It is easy to see that provides an assessment of the slope of at . Intuitively, for an SOEC instance with a large (i.e., well conditioned case), a root-finding method will be able to move towards the root of faster compared to an instance with a small (i.e., ill-conditioned case). See Figure 2.1 of Lin et al. (2018b) for a graphical illustration of this statement.
Theorem 2.5
Given an input tuple , suppose and . Algorithm 1 generates a feasible solution at each iteration with a probability of at least . Moreover, it returns a relative -optimal and feasible solution with this probability in at most
[TABLE]
calls to oracle .
The bound on the number of oracle calls increases with because both the step-length and the optimality tolerance decrease with . The maximum number of oracle calls is also a decreasing function of both the condition measure and tolerance , that is, SFLS requires fewer iterations for problems that are better conditioned and when and are larger. Here, both and require knowledge of , which is difficult to compute exactly. We want to point out that the dependence of and on are introduced here only to simplify the theorem and its proof, which helps readers to understand the main idea behind our technique. In §5, we will show that SFLS has a similar complexity even if in and is replaced by an upper bound with and can be computed (by Algorithm 4) in a low cost independent of .
SFLS relies on the availability of a valid stochastic oracle . Standard subgradient methods cannot be used as oracles to solve (2) since computing a deterministic subgradient of requires exact evaluations of for (see Bertsekas 1999 or Danskin 2012, p.737), which is challenging due to the high-dimensional expectations in the definition of these functions. Indeed, the expectation in each can be replaced by a direct SAA to obtain a sampled version of . This replacement is also problematic as subgradients of provide biased subgradients of due to the maximization in the definition of the latter function.
To avoid this issue, we reformulate (2) into the equivalent min-max (i.e., saddle-point) form
[TABLE]
where and . Given , it is easy to check that can be chosen as a unit vector with 1 corresponding to an index and zeros for the remaining indices. Let , , and , where to ease notation we suppress the dependence of and on the level parameter since it is always equal to a fixed value when these functions are invoked. Therefore, (4) can be reformulated as
[TABLE]
Let be an SAA of . Subgradients of provide an unbiased estimate of subgradients of because there is no nonlinear operator (e.g., maximization) acting on the expectation defining . The oracles that we discuss for SFLS in §§3-4 will thus solve (5).
3 Idealized Stochastic Oracle
In §3.1, we present stochastic mirror descent (SMD) in the form a stochastic oracle. In §3.2, we establish that SMD is indeed a stochastic oracle that can be used in SFLS (i.e., Algorithm 1) and then highlight computational issues that prevent its use. The discussion here serves a dual role. First, it provides practical motivation and sets the stage for developing a tractable stochastic oracle in §4. Second, it provides basic concepts on primal-dual methods needed throughout the paper, also making the paper more accessible to readers potentially unfamiliar with such methods.
3.1 Stochastic Mirror Descent
Stochastic mirror descent (SMD) (Nemirovski et al. 2009) is a well-known primal-dual method for solving saddle-point problems such as (5). SMD updates primal and dual variables and of (5), respectively, by employing stochastic subgradients of and a projection operator. Let for , where is the subgradient operator. We denote the stochastic subgradient vector of by
[TABLE]
The projection employed by SMD relies on a distance function, known as Bregman divergence, that has as its argument and operates over . The space is equipped with a convex and continuously differentiable distance generating function modulus 1 and a set of nonzero subgradients . The Bregman divergence expressed using is
[TABLE]
The projection operator (or prox-mapping), for any , and , is defined as .
Algorithm 2 summarizes the steps of SMD presented in the form of a stochastic oracle. The inputs to this algorithm are a level parameter , an optimality tolerance , a probability , an iteration limit (we specify this limit later in Proposition 3.1), and a step-length rule for all . Line 2 sets the initial solution . Algorithm 2 executes lines 4 and 5 for iterations. At iteration , line 4 constructs a stochastic subgradient using a sample of the random variables underlying the expectations in the objective and constraints of (1). Line 5 computes a step-length weighted average of past solutions. It also uses the stochastic subgradient computed in line 4 and a projection operator to find an updated solution . After exiting the for loop, line 7 uses the averaged primal solution to compute an upper bound on . The pair is returned in line 8.
It is worth noting that the update in line 5 relies on subgradients of an SAA (with a single sample), which provides unbiased subgradients of , unlike the biased subgradients that arise when working with SAAs of in the primal problem (2). In other words, a key benefit of the primal-dual reformulation (4) is that its objective allows the computation of unbiased subgradients after using SAAs to replace exact expectations.
3.2 Validity of Stochastic Oracle and Computational Issues
We analyze below the validity of SMD as a stochastic oracle and also discuss its computational tractability. Our analysis, based on Nemirovski et al. (2009), requires specifying the distance generating function introduced in §3.1 and stating a standard assumption.
To define , we equip and with their own distance-generating functions modulus with respect to norm and modulus with respect to norm . This means that is -strongly convex, continuous on , and continuously differentiable on the set of non-zero subgradients . Similarly, is - strongly convex, continuous on , and continuously differentiable on . Typical choices for and are and , respectively. In addition, it is common to set and . Defining the diameters of the sets and as and , the distance-generating function associated with is
[TABLE]
Next, the following standard assumption is needed to analyze SMD as well as other methods in the rest of the paper. Denote by expectation of the -dimensional vector , that is, a deterministic subgradient. Moreover, let and represent the dual norms of and , respectively. {assumption} For any , there exist for such that is well defined and satisfies
[TABLE]
where and represent the sub-differentials with respect to and , respectively. Moreover, there exist positive constants , and such that
[TABLE]
for any and , which indicate that and have a light-tailed distribution and their moments are bounded.
Proposition 3.1 presents the iteration complexity of SMD, which follows from results in Nemirovski et al. (2009), and in addition, establishes that SMD is a valid stochastic oracle, that is, it satisfies Definition 2.2. The proof of this proposition relies on establishing that the primal-dual gap is guaranteed to be less than a given with a probability of at least for a given , where and is computed in Algorithm 2. We also require the following constants:
[TABLE]
Proposition 3.1
Given an input tuple , the SMD solution satisfies with probability at least in at most
[TABLE]
gradient iterations. As a consequence, SMD is a valid stochastic oracle with .
When solving (5), the dependence of the iteration complexity on in Proposition 3.1 has an additional term compared to the known SMD complexity dependence of for solving an unconstrained version of this problem. Moreover, the analogous complexity dependence on inside logarithmic terms (see definition of ) in this proposition is comparable to the unconstrained case.
We note that SMD is a valid stochastic oracle, exhibits a favorable iteration complexity, and is based on unbiased subgradients of . Nevertheless, SMD is not directly implementable because the upper bound is challenging to compute exactly as the definition of embeds expectations. Replacing these expectations by an SAA leads to a biased estimate of the upper bound . This bias can be reduced by using a large number of samples but doing this would lead to an approach with high data complexity, which we would like to avoid. In other words, although our saddle-point formulation facilitates the computation of unbiased subgradients needed by SMD to obtain a near optimal and high probability feasible solution, its upper bound , which serves as the constant returned by the oracle (see Definition 2.2), cannot be computed.
The aforementioned bound computation challenge is further exacerbated if one wishes to change the stopping criterion of Algorithm 2 (i.e., line 3) from a maximum iteration limit to a bound on the primal-dual gap . In the latter case, implementing SMD would also entail the computation of the lower bound , which suffers from analogous bias and data complexity issues when expectations in its definition are replaced by SAAs. In addition, the optimization problem over in the definition of is in general a high-dimensional non-smooth convex optimization problem and solving such a problem multiple times is computationally burdensome. Therefore, it is apriori unclear how one should go about designing a computationally tractable oracle to overcome these issues and what the iteration complexity of such an oracle would be.
4 Tractable Stochastic Oracle
In this section, we design a computationally viable stochastic oracle by combining SMD and an online validation technique (Lan et al. 2012), and in particular, extending the latter technique originally proposed for minimization problems to handle min-max saddle point problems. This oracle overcomes the issues highlighted at the end of §3.2 by defining bounds that are (i) tractable to compute with low data complexity and (ii) do not suffer from the bias issue when replacing expectations in their definitions by SAAs, as was the case with the bounds and . We present our algorithm in §4.1 and prove that it is a stochastic oracle in §4.2, where we also analyze its complexity.
4.1 Online Validation Based Stochastic Mirror Descent
Algorithm 3 contains the steps of our proposed online validation based stochastic mirror descent (OVSMD) scheme, which differs from Algorithm 2 only in line 7, where the upper bound on is replaced by . The quantity is an approximation of the following upper bound obtained using the online validation technique:
[TABLE]
This upper bound holds because
[TABLE]
where the first inequality is true because is a subgradient with respect to of the function , which is concave in , and the second inequality follows directly from the convexity of in . Therefore, we have
[TABLE]
that is, is an upper bound on , albeit potentially weaker than . Computing requires the exact evaluations of , and , which are not in general available because they involve expectations. In contrast, the term computed in line 7 of Algorithm 3, which is stochastic approximation of , can be easily computed in an online manner by solving a simple linear optimization problem.
As discussed in §3.2, replacing the iteration limit based stopping criterion by one that approximates an optimality gap requires a lower bound on . Following a similar argument to the upper bounding case above, we define the lower bound
[TABLE]
Since is convex in , it follows that . Although is in general a weaker lower bound than , the former bound is computed by solving a linear optimization problem as opposed to the potentially challenging non-smooth convex optimization problem defining the latter bound. Finally, we employ an online validation based approximation of to avoid computing expectations and obtain
[TABLE]
Despite the computational tractability of and , these are stochastic quantities and subject to noise. Hence they do not always provide valid bounds on . In §4.2, we show that and are nevertheless sufficiently close to with high probability after a finite number of iterations (see Theorem 4.2).
4.2 Validity of Stochastic Oracle and Iteration Complexity
We establish here the validity of OVSMD (i.e., Algorithm 3) as a stochastic oracle and derive its iteration complexity. Proposition 4.1 contains the two main ingredients underlying the analysis of OVSMD. Part (i) of this proposition shows that for a given the inequality holds with high probability when is sufficiently large. In other words, the deterministic quantities and computed using the OVSMD solutions provide “good” deterministic estimates of the level set function . This is not directly useful since OVSMD can only compute stochastic approximations of these quantities, as already discussed in §4.1. Part (ii) of Proposition 4.1 establishes that and are respectively close stochastic approximations of and at convergence with high probability. It then follows that the quantities and are “good” stochastic estimates of the level set function, and in particular, allows OVSMD to be used as a stochastic oracle.
Proposition 4.1
Given an input tuple , OVSMD computes , such that:
- (i)
The inequality holds in at most
[TABLE]
gradient iterations.
- (ii)
The inequalities and hold in at most
[TABLE]
gradient iterations.
Leveraging Proposition 4.1, Theorem 4.2 shows that OVSMD is a valid stochastic oracle and also presents its iteration complexity.
Theorem 4.2
Given an input tuple , the OVSMD guarantees and with probability at least in at most
[TABLE]
gradient iterations. As a consequence, OVSMD is a valid stochastic oracle with .
Despite OVSMD being a tractable oracle, the dependence of its iteration complexity on both and is identical to the analogous dependence seen with the idealized SMD oracle analyzed in Proposition 3.1. Moreover, in terms of , OVSMD is only a worse than the known complexity of SMD in the unconstrained case, where feasibility is not a concern.
5 SFLS with OVSMD as its Stochastic Oracle
In this section, we provide theoretical support for the use of OVSMD as SFLS’s stochastic oracle in §5.1 and then discuss implementation guidelines in §5.2.
5.1 Theoretical Analysis
Theorems 2.5 and 4.2 can be used to derive the (gradient) iteration complexity of SFLS when using OVSMD as the stochastic oracle. We state this complexity in Corollary 5.1.
Corollary 5.1
Given an input tuple , let and . Moreover, suppose OVSMD with is chosen as the stochastic oracle . Then SFLS returns a relative -optimal and feasible solution with probability of at least using at most
[TABLE]
OVSMD calls and
[TABLE]
gradient iterations.
This complexity result is somewhat idealistic because the inputs to SFLS, namely and , require knowledge of , which is difficult to compute exactly. A possible resolution is to compute an upper bound on , denoted by , such that . If is much smaller than , then the optimality tolerance will be substantially more stringent and thus lead to a larger complexity than the iteration bound in Corollary 5.1. Therefore, to obtain a complete theoretical assessment of the computational complexity of SLFS with OVSMD, it is important to incorporate the cost of finding a that is comparable to (i.e., ).
Fortunately, OVSMD can itself be used to compute the desired . We discuss the intuition behind its use for this purpose and then formally state the result. Recall that since . We consider obtain an upper bound by solving (2) with and a small enough optimality gap. By Theorem 4.2, OVSMD with can guarantee with high probability. This suggests setting . However, it is a priori unclear how small should be in order to ensure and . Therefore, we run OVSMD multiple times, starting from a tolerance , geometrically reducing this tolerance after each run, and stopping this procedure once and hold. We can then use Theorem 4.2 and the condition to show that , which implies . We formalize the aforementioned approach in Algorithm 4.
Theorem 5.2 establishes the complexity of employing Algorithm 4 to compute and subsequently running SFLS leveraging this computation.
Theorem 5.2
Given an input tuple , suppose we compute using Algorithm 4 and then execute SFLS to find a relative -optimal and feasible solution with a probability of at least using , , and OVSMD with as the stochastic oracle . This procedure requires in total at most
[TABLE]
OVSMD calls and
[TABLE]
gradient iterations.
Theorem 5.2 provides a realistic theoretical assessment of the computational burden of solving SOECs using SFLS. Interestingly, it shows that running Algorithm 4 to compute before executing SFLS and replacing the unknown term in the definitions of and with the computed value does not change the overall big- oracle and gradient iteration complexities in Corollary 5.1, except for logarithmic terms.
The complexity of SFLS (combined with OVSMD) in Theorem 5.2 is comparable in terms of its dependence on and to the complexity of the algorithm in Yu et al. (2017), which does not ensure feasibility. This suggests that our procedure is efficient at ensuring feasibility. The cost of ensuring feasibility, however, appears in the dependence of the SFLS iteration complexity on the condition measure . Such dependence is absent in approaches that do not ensure feasibility.
Another relevant comparison is with the deterministic feasible level set approach (DFLS) of Lin et al. (2018b) and its variant in Lin et al. (2018a), which are both applicable to solve deterministic constrained convex optimization problems. The complexity of DFLS based methods depend on the number of data points that define expectations and thus lead to large data complexity, and in particular, have infinite complexity when expectations are defined over continuous random variables. In contrast, the complexity of SFLS in Theorem 5.2 does not depend on the number of data points. In addition, compared to DFLS, the iteration complexity of SFLS has only additional logarithmic factors involving and , which is encouraging, as the stochastic level set algorithm (i.e., Algorithm 1) and OVSMD oracle need to contend with several challenges that arise due to the presence of expectations in SOECs.
In summary, our theoretical analysis of SFLS and comparison with known complexities of state-of-the-art approaches suggests that SFLS is effective in terms of iteration complexity at computing a high probability feasible solution path for SEOCs, a much broader and challenging class of problems than deterministic constrained convex programs. Moreover, a fully stochastic approach such as SFLS is theoretically necessary to achieve low data complexity in this context.
5.2 Implementation Guidelines
As is common with first-order methods, the implementation of SFLS requires parameter tuning. A direct implementation of SFLS in a manner consistent with Theorem 5.2 requires selecting , , , and ; estimating constants and (needed to define in OVSMD); and then computing . While these parameters can be estimated or approximated, we suggest a simpler implementation strategy that largely side-steps such tuning. Firstly, we avoid stopping SFLS by pre-specifying optimality tolerance and instead stop it based on an outer iteration limit. This is possible because the SFLS outer iterations only affect the suboptimality of the incumbent feasible solution, that is, being a feasible level set method, SFLS can return feasible and implementable solutions when terminated after any number of outer iterations. Secondly, instead of choosing the number of inner iteration in OVSMD as based on pre-specified and , we directly specify . According to (16), is strictly monotonically decreasing in and thus in so that a relative -optimal and feasible solution with can be guaranteed. Corollary 5.3 establishes that the convergence of SFLS in this implementation.
Corollary 5.3
Suppose we have an input tuple and the iteration limit in OVSMD is . Given , SFLS finds a relative -optimal and feasible solution with and with a probability of at least using at most OVSMD calls and gradient iterations.
Overall, following the aforementioned strategy only requires the choice of , , and – a significant reduction in implementation burden.
For choosing and , we consider a discrete set of values and tune the algorithm, that is, we test the performance of SFLS for a few iterations or data passes for each value, and select the one that leads to the largest decrease in suboptimality. Selecting is easy when an initial feasible solution is available because we have . In this case, we estimate using an SAA and then set to a larger value to account for approximation error and ensure we have . If a feasible solution is not readily available, we can find one by applying a minor modification of Algorithm 4 to solve
[TABLE]
which does not include the term in (4) corresponding to , that is, . Finally, the step length can be specified as for a given constraint , which is tuned. While is chosen as in our theoretical analysis to simplify proofs, analogous results hold for a generic constant . We omit these general results for the sake of brevity as they do not change the dependence of our iterations bounds on , , and .
6 Numerical Experiments
In this section, we evaluate the numerical performance of SFLS on three diverse SOEC applications: (i) approximate linear programs for solving Markov decision processes, (ii) multi-class Neyman-Pearson classification, and (iii) learning with fairness constraints. SOECs in the first application contain expectations of continuous random variables while those in the second and third applications involve discrete random variables. Our first algorithmic benchmark is the stochastic subgradient method YNW of Yu et al. (2017) as it is the only first order approach (we are aware of) that can handle SOECs with multiple constraints. In addition, we also compare against the deterministic feasible level-set method (DFLS) of Lin et al. (2018b) because it ensures a feasible solution path. Specifically, comparing SFLS and DFLS allows us to evaluate the benefits of the reduced data complexity in our stochastic approach. In §6.1, we describe our computational setup and then the performance of algorithms on applications in §§6.2-6.4.
6.1 Computational Setup
We implemented SFLS, DLFS, and YNW in Matlab running on a 64-bit Microsoft Windows 10 machine with a 2.70 Ghz Intel Core i7-6820HQ CPU and 8GB of memory. We set and in all three algorithms. We followed the guidelines in §5.2 when implementing SFLS and thus had to choose only , , and . We based on the solution . We tuned over the discrete set and over the discrete set . We selected and tuned over the set of possible values . We employed a mini-batch technique to construct the stochastic gradients in SFLS and YNW.
Similar to SFLS, DFLS solves the subproblem approximately in the th outer iteration and uses the returned solution to update as . Following Lin et al. (2018b), we use the standard subgradient descent method to solve this subproblem and the parameters and and the inner iteration limit in DLFS are tuned in the same way as in SFLS as described above. To apply DLFS, we constructed a deterministic version of each SOEC using SAAs of expectations. We found, consistent with Lin et al. (2018b), that using SAAs in lieu of expectations over continuous random variables in the perishable control problem (first application) did not sufficiently represent the original problem even when using a large number of samples. We thus omitted DFLS as a benchmark for this application. This was not an issue for the remaining two applications because expectations are defined over discrete random variables. To avoid the quality of SAAs confounding our performance evaluation, we chose instances for these two applications such that expectations can be evaluated exactly, albeit requiring more time.
We followed the guidance in Yu et al. (2017) to setup YNW. Specifically, we chose the control parameters and as and , respectively, as a function of the total number of iterations , where is the weight of the gradient of the objective function and is the weight of the proximal term in the updating equation of in YNW. Similar to SFLS, we used a mini-batch technique to construct the stochastic gradients and evaluate the objective values.
6.2 Approximate Linear Programming for Markov Decision Processes
Approximate linear programs (ALPs) address the well-known curse of dimensionality associated with directly solving large-scale Markov decision processes (MDPs; Puterman 1994) by computing a value function approximation. We illustrate how our SFLS method can be applied to tackle ALPs, and thus large-scale MDPs, by considering a challenging perishable inventory control problem with partial backlogging and lead time. We begin by presenting the MDP for this problem and refer the reader to Lin et al. (2019) for its derivation and detailed application context.
Consider the management of orders for a single product with a finite life time of periods and an order lead time of periods, that is, the product takes periods to be delivered from when it is ordered and periods to perish from receipt. The state space of the MDP is represented by the vector
[TABLE]
where , , denotes the order quantities that will be received periods from now, and , , the on-hand inventory with periods of lifetime remaining. The order quantity is at most and belongs to the interval , which implies for and for . The element of the state is bounded below by to allow limited or partial backlogging, that is, any units backlogged beyond are lost sales. To ease exposition, we write and to capture the state and action domains, respectively, and use to represent the initial state. Assuming orders are served on a first-come-first-serve basis, the MDP state transitions as
[TABLE]
where represents stochastic demand with distribution . Moreover, the cost associated with ordering at state is
[TABLE]
where the per unit lost sale, disposal, purchasing, holding, and backlogging costs are , , , , and , respectively; is taken over ; and is a discount factor. The infinite horizon (discounted cost) MDP formulated using the aforementioned components can be solved using the fixed point equations
[TABLE]
ALPs approximate the high-dimensional MDP value function (Schweitzer and Seidmann 1985, de Farias and Van Roy 2003) using a linear combination of basis functions. We construct the ALP value function approximation using an intercept and basis functions , , that is, , where is the basis function weight vector. It is common to require that the pair belongs to a compact set . The VFA weights are computed by solving
[TABLE]
The feasibility of the ALP constraints is important because it ensures that the objective function of a feasible solution provides a lower bound on the optimal policy value, which can be used to assess the suboptimality of heuristic policies (see, e.g., Proposition 4 in Adelman and Mersereau 2008). Thus, in principle, methods to solve ALP would benefit from emphasizing feasibility as we do in SFLS.
Since the linear program above is semi-infinite, constraint sampling is a popular strategy to approach its solution and obtain a high-probability feasible solution (de Farias and Van Roy 2004). Specifically, suppose we sample state-action pairs . The ALP with constraints corresponding to these samples takes the form of (1):
[TABLE]
We solve this linear program in our experiments.
Following Lin et al. (2019), we constructed instances with and , chose to be a truncated normal in the interval with mean and the standard deviation 2, and fixed , , , , , and equal to , , , , , and , respectively. We experimented with three instances based on the triple being equal to , , and . We employed eighteen basis functions (): , , , and , where and are the 25-th and 50-th quartiles of the demand distribution. The domain for the basis function weights was taken to be the box . We chose as 500.
In all methods, we use the initial solution with and which is feasible for (6.2). Our SFLS implementation uses , , and the step length rule . We do not report results for DFLS because, as alluded to in §6.1, obtaining a good deterministic approximation using SAAs is non-trivial for the perishable inventory control problem. We use a mini-batch technique with a batch size of to construct stochastic estimates of the gradients and function values of , in both SFLS and YNW. In SFLS, we choose the numbers of inner (i.e. ) and outer iterations to be and , respectively, which leads to stochastic gradient steps (inner iterations) in total. Hence, we choose the total number of iterations in YNW as so that both methods evaluate the same number of stochastic gradients in total which lead to similar runtime (about 1100 seconds).
Figure 1 displays the performance of SFLS and YNW. The -axes of the top subfigures report the optimality gap while these axes in the bottom subfigures show the feasibility of solutions by plotting . Here, for are calculated by approximating the expectations in their definitions in (6.2) with samples of demand . The optimal value is approximated by the objective value found by a separated run of SFLS with sufficient iterations ( outer and inner iterations). We track these measures as a function of the number of iterations performed by each algorithm in the -axis. To indicate the values of and corresponding to the high probability feasible solutions maintained at each SFLS (outer) iteration we use line markers in Figure 1. The YNW curves have no line markers as there are no outer iterations ensuring feasibility. SFLS finds a feasible solution quickly and maintain a relatively large constraint slack but YNW does not always ensure feasibility. SFLS also reduces the suboptimality of solutions faster, suggesting that SFLS is able to balance optimality and feasibility well on these instances.
6.3 Multi-class Neyman-Pearson classification
Another application that gives rise to (1) is Neyman-Pearson classification. In multi-class classification, there exist classes of data, where , , denotes a random variable defined using the distribution of data points associated with the -th class. To classify a data point to one of the classes, we rely on the same number of linear models , . The predicted class for is . High classification accuracy in this scheme requires with to be large and positive (Weston and Watkins 1998, Crammer and Singer 2002), that is, the classifiers have discriminatory power. Minimizing the expected loss is one approach to promote this goal, where is a non-increasing convex loss function and is expectation taken over .
Suppose misclassifying has a cost that depends on but not on the predicted class. We propose a model that prioritizes classes with relatively higher misclassification costs using constraints and simultaneously trains the set of linear models by solving
[TABLE]
where it is assumed (without loss of generality) that class has the highest misclassification cost and the value of is chosen to capture the misclassification cost of class . Here is a regularization parameter. This formulation can be easily extended to handle the case where the mis-classification cost depends on both the true and predicted classes. Indeed, (18) is of the form (1). Infeasible solutions may result in large misclassification costs for some classes, which is undesirable, and creates a need for methods that emphasize feasibility.
We created test instances using the multi-class classification LIBSVM datasets connect-4, covtype, and news20 from Chang and Lin (2019). We selected these instances as their size still allows us to run DFLS in the manner discussed in §6.1. We summarize in Table 1 the number of classes, the number of data points in each class, and the number of features in these four datasets. We chose the loss function (18) to be the hinge loss . Let follow the empirical distribution over the dataset of class for , which implies that all the expectations in (18) become finite-sample averages over data classes. We set the parameters and for .
In all methods, the solution is used as the initial solution and it is feasible for (18). To apply SFLS and DFLS, we chose , , and across all datasets. Note that for (18). In DFLS, we solve subproblems via standard subgradient descent method. In SFLS and DFLS, we choose step size for connect-4 and covtype and choose for news20. Both SFLS and YNW employed a mini-batch size of to construct the stochastic gradients and the objective values. We chose the number of iterations in YNW so that its total number of data passes is for connect-4 and news20 and for covtype. Then, we also terminated SFLS and DFLS when the total data passes they performed exceed YNW.
Figure 2 displays the performance of each method. The -axes of the first row reports the term , that is, it focuses on optimality, while this axis in the second row shows the feasibility of solutions by plotting . We track these measures as a function of the number of equivalent data passes performed by each algorithm in the -axis, where a data pass involves going over the number of data points equal to the size of the training data. This is possible since the expectations in our instances are over discrete random variables. Tracking data passes allows us to assess algorithms in terms of data complexity. Similar to Figure 1, we uses line markers to indicate the values of and corresponding to the solutions maintained at each SFLS outer iteration, while YNW has no line marker since it does not maintain feasibility. Since DFLS needs two data passes in each inner iteration, it can only perform one or two outer iterations with the number of data passes in Figure 1. Hence, for a better visualization, we use line markers to also indicate the inner iterations of DFLS instead of only outer iterations. In this figure, is approximately by the objective value returned by DFLS after a sufficient number of data passes (i.e. at least data passes with inner iterations.)
On the connect-4 data set, SFLS maintains feasibility and reduces the optimaliy gap quite rapidly after a few data passes. Interestingly, despite providing an initial feasible solution, YNW decreases the optimality gap at the beginning by moving to a highly infeasible solution. The peformance of both methods on the covtype data are comparable. On the news20 data set, SFLS provides feasible solutions with smaller optimality gaps sooner than the benchmark method. The comparison of SFLS and YNW highlights the advantage of SFLS in terms of feasibility. Specifically, efficient methods that do not emphasize feasibility could lead to highly infeasible solutions if terminated prematurely (e.g., the connect-4 dataset).
DFLS also maintains a feasible solution path on all the datasets, as expected. However, its optimality gap reduces at a much slower rate with the number of data passes compared to SFLS because it uses deterministic subgradients based on the entire data set. These results thus underscore the importance of developing methods, such as SFLS, with low data complexity to balance optimality and feasibility.
6.4 Learning with Fairness Constraints
We consider learning a classifier with fairness constraints. Other examples include training predictive models with constraints on coverage rates, churn rates, and stability. Please see Goh et al. (2016) for further motivation and a non-convex formulation. Here we provide a convex formulation for these problems, which can be viewed as a tractable relaxation of the version in Goh et al. (2016) that admits the SOEC structure (1).
Suppose is a data point from a distribution , where is a feature vector and is the class label. Let and denote two different distributions of features (that are not necessarily labeled), which may represent male and female individuals. The goal is to train a classifier that minimizes classification loss. The correct classification of data vector implies that . One can train such a classifier subject to fairness constraints by solving
[TABLE]
where is a regularization parameter, is a constant, is a non-increasing loss function,
[TABLE]
and represents the probability of the (random) classifier predicting as positive. Therefore, and represent the percentages of instances in and predicted as positive, respectively. The first constraint guarantees that the percentage of the positively predicted instances in is at least a fraction of that in . The second constraint has similar interpretation. An analogous model was considered in Goh et al. (2016) but it involves non-convex constraints.
Observing that , we can reformulate the first constraint as and approximate by so that we obtain a convex constraint . Applying an analogous convex approximation to the second constraint, we obtain the following convex formulation for training a classifier subject to fairness constraints:
[TABLE]
The left hand side of the first constraint will be large if the classifier is not “fair”, that is, it makes very negative for most of from but very positive for most of from . Similarly, the left hand side of the second constraint will be large if the model makes very positive for most of from but very negative for most of from . Choosing an appropriate ensures that the obtained model is fair to both and . Indeed, a solution that violates constraints in this formulation translates to a classifier that discriminates against one of the two classes.
For testing, we considered the “a9a” dataset, also used by Goh et al. (2016) and another dataset dubbed “LoanStats” from Lending Club (2019). We chose , , and in each case. The distributions , , and were defined as empirical distributions based on each dataset as described below. The goal in the a9a dataset is to predict people making more than 50,000 USD. Following Goh et al. (2016), we used the 32,561 training instances () and the 16,281 testing instances in the dataset to construct the objective function and constraints, respectively. Since we need male and female subsets to construct constraints, we further split the testing data into 14,720 male instances () and 1,561 female instances (). The LoanStats dataset contains information of loans issued in the fourth quarter of 2018 and the goal is to predict if a loan will be approved or rejected. After creating dummy variables, each loan is represented by a feature vector of 250 dimensions. We randomly partitioned the dataset into a set of loans () used to construct the objective function and a set of loans used to build the constraints. We further split the second set based on whether the feature “homeOwnership” equals “Mortgage” () or some other value () to obtain 31,966 and 32,519 loans in two subsets, respectively.
All methods are initialized at , which is feasible for (18). In SFLS and DFLS, we chose , and across all datasets. Note that . In SFLS, we chose and for a9a and LoanStats datasets, respectively. In DFLS, we chose and for a9a and LoanStats datasets, respectively. Both SFLS and YNW employed a mini-batch size of and for a9a and LoanStats datasets, respectively. Similar to §6.3, we chose the number of iterations in YNW so that its total number of data passes is 300. Then, we also terminated SFLS and DFLS when the total data passes they performed exceed .
Figure 3 displays the performance of SFLS, YNW, and DFLS as a function of data passes. The interpretation of the axes and line markers in this figure are analogous to the ones in Figure 2. In this figure, is approximately by the objective value returned by DFLS after a sufficient number of data passes (i.e. at least data passes with inner iterations.) On the a9a dataset, SFLS maintains a feasible solution path, as expected, while the YNW solutions are initially infeasible and become feasible with more data passes. The YNW reduces optimality gap more rapidly at the beginning while SFLS catches up quickly. The objective function value of YNW cannot be interpreted as an optimality gap when its solutions are infeasible since the corresponding objective function value can be super optimal. This feature is clearly visible on the LoanStats data. Here most of the YNW solutions are infeasible and superoptimal, that is, is non-positive. The SFLS solution path continues to be feasible and suboptimal on this dataset, with its suboptimality decreasing consistently after each outer iterations. DFLS also produces a feasible path but does not effectively reduce the optimality gap because its data complexity is high, that is, it requires a large number of data passes to achieve a small optimality gap. Similar to §6.3, we once again find that the low data complexity of SFLS is critical to balance optimality and feasibility when solving an SOEC.
7 Conclusion
We consider constrained optimization models where both the objective function and multiple constraints contain expectations of random convex functions. These models, referred to as stochastic optimization problems with expectation constraints (SOECs), arise in several machine learning, engineering, and business applications. We develop a stochastic feasible level-set method (SFLS) to solve SOECs, propose a tractable oracle to be used with SLFS, and analyze related iteration complexities. SFLS’s total iteration complexity is comparable to stochastic subgradient methods in terms of but depends on a condition number – the cost of requiring feasibility. We evaluate the performance of SFLS across three applications involving approximate linear programming, multi-class classification, and learning classifiers with fairness constraints. We find that SFLS exhibits key advantages over existing methods. First, it ensures a feasible solution path with high probability while an existing state-of-the-art stochastic subgradient method can return highly infeasible solutions when terminated before conservative termination criteria are met. Infeasibilities may void the use of a solution in practice, especially if constraints model implementation requirements. Thus, the ability of SFLS to compute feasible solutions before convergence is practically relevant. Second, SFLS computes feasible solutions with small optimality gaps using only a few data passes owing to its low data-complexity, which is a desirable property when expectations are defined using large datasets that are expensive to scan. In contrast to SFLS, a recent deterministic feasible level set method exhibits high data complexity and large optimality gaps. Our theoretical and numerical findings bode well for the use of SFLS to solve SOECs and motivates further research into stochastic first order methods that emphasize feasibility.
8 Proofs of Theoretical Results
In this section, we provide the proofs of all technical results in the paper.
Proof 8.1
*Proof of Lemma 2.3: *** Since , it follows from Lemma 2.1(c) that . Therefore, since we have . Moreover, by Definition 2.2, we have with probability of at least , which implies that is a feasible solution to (2) since . \halmos
Proof of Theorem 2.5 depends on the following lemma.
Lemma 8.2
Given an input tuple , a stochastic oracle with returns and such that with probability of at least .
Proof 8.3
Proof. The inequality holds by definition of . By definition of stochastic oracle (Definition 2.2) and the property of , it follows that , , and hold with probability of at least . Since , Lemma 2.1(c) implies that . Therefore, using the inequality we get and since . Finally, combining the inequalities and (or equivalently ), we get . \halmos
In the proof of Theorem 2.5 we need the following property of the condition measure . In particular, it can be easily verified from the convexity of and for any (Lemma 2.3.5 in Nesterov 2004) that is monotonically increasing in on and
[TABLE]
Proof 8.4
Proof of Theorem 2.5:* We first show that the Algorithm 1 generates a feasible solution at each iteration with high probability. Let be the largest value of such that and the following inequality holds:*
[TABLE]
Notice that since and . It follows from Lemma 8.2 that with a probability of at least we have,
[TABLE]
Since , we have
[TABLE]
and
[TABLE]
with a probability of at least , where the last inequalities in both (23) and (24) follow from (20). Inequality (23) and the condition imply that . Applying this argument recurrently and using the fact that , we have (23), (24) and holds for . Therefore, since for , Lemma 2.3 implies the solution generated at iteration is feasible to (1) with a probability of at least . We next show that (21) holds with a high probability until Algorithm 1 terminates. By the definition of , we know that (21) is violated when , i.e. . Since and is monotonically increasing, we can show that
[TABLE]
where the last inequality holds by (23). Using the definition of , (25), and (22) for (specifically, ), we have
[TABLE]
which indicates that Algorithm 1 must stop before . Therefore, SFLS generates a feasible solution with a probability of at least at each iteration before termination.
We now proceed to establish that the terminal solution of SFLS is relative -optimal solution. By definition of and (22) it follows that for all . Hence,
[TABLE]
Combining (26) and derived from (20) stipulates that with a probability of at least :
[TABLE]
where we used (22) in the second inequality. Hence, at termination of Algorithm 1 we get since the algorithm stops when .
Finally we show that . By recursively applying inequality (24) we get
[TABLE]
with probability of at least , which implies for the choice of . Hence, we have where the first inequality follows by (22), the second by (20), and the third by (27). This indicates that the stopping criterion of Algorithm 1 holds with a probability of at least when and SFLS requires at most calls to oracle . \halmos
Proof 8.5
Proof of Proposition 3.1:* The proof of the first part directly follows from Proposition 3.2 in Nemirovski et al. 2009. We only show that SMD is a valid oracle. It is straightforward to see that the inequality implies , where the first inequality holds since is an upper bound on and the second since is a lower bound on . This indicates that the conditions provided in Definition 2.2 are satisfied. \halmos*
To show part (i) of Proposition 4.1, we use known lemmas 8.6 and 8.7 as well as prove lemmas 8.8 and 8.10. To prove part (ii) of this proposition we need Lemma 8.12. Before stating these lemmas, we present some required notation and representations, which we present next. We denote the diameter of with respect to by
[TABLE]
In addition, for any , , , , and , it is easy to verify for that
[TABLE]
where and .
Lemma 8.6** **(Equation (2.37) and Lemma 6.1 in Nemirovski et al. 2009)
Let , be a set of random variables, and for . For any and , we have
[TABLE] 2. 2.
Let , be a set of random variables, and for . For any and , we have
[TABLE] 3. 3.
Let , be a set of random variables, and for . For any and , we have
[TABLE]
Lemma 8.7** **(Lemma 2 in Lan et al. 2012)
Let and for be respectively a sequence of i.i.d. random variables and deterministic numbers; ; the conditional expectation conditioning on for ; and be a measurable function of such that either
- Case A:* and , or*
- Case B:* ,*
*almost surely for all . Then for any , we have the followings:
In case A:*
[TABLE]
In case B:
[TABLE]
where .
Lemma 8.8 shows that the stochastic subgradient has a light-tailed distribution and bounds the Bregmann distances. Define
[TABLE]
Lemma 8.8
The following inequalities hold:
[TABLE]
Moreover, when , we have
[TABLE]
Proof 8.9
Proof. Applying Jensen’s inequality and using the definitions of , , and the inequalities (8) and (9), we have
[TABLE]
Using (36) and Jensen’s inequality, it follows that
[TABLE]
Hence, we have
[TABLE]
where the first inequality follows from the definition of and the inequality for any , the second from (37), the third from Jensen’s inequality for concave functions, and the fourth by inequalities (8) and (9). Following a similar argument, we can also show that and . Finally, inequalities (33), (34), and (35) follow because , and are modulus , and , respectively. \halmos
Lemma 8.10
Let . Given , Algorithm 3 computes , such that
[TABLE]
Proof 8.11
Proof.Since and in Algorithm 3, by Lemma 8.6 we have, for any ,
[TABLE]
where the second inequality follows by (35). In addition, by definition of , for any we have
[TABLE]
Applying (8.11) to (41) and reorganizing terms lead to
[TABLE]
Maximizing both sides of the above inequality over implies
[TABLE]
Let and for . From Lemma 8.6 it follows that for any ,
[TABLE]
Rewriting and applying (43) to (42) yield
[TABLE]
We next find a probabilistic bound for the right hand side of the above inequality.
Bound on :* By our choice of , i.e. and (35), for any we have*
[TABLE]
Define and . Because is independent of and , we have . In addition, it can be verified that where the second inequality holds by (45). Using this inequality and (38), we get . Hence, it follows from Case A in Lemma 8.7 that
[TABLE]
Bound on :* Define \psi_{s}:=\gamma_{s}\nu_{s,t}\big{(}\left\|G({\mathbf{x}}^{(s)},{\mathbf{y}}^{(s)},{\bm{\xi}}^{(s)})\right\|_{*,z}^{2}+\left\|\Delta_{s}\right\|_{*,z}^{2}\big{)} and . We then have*
[TABLE]
where the first inequality is from Jensen’s inequality and the second inequality is from (36) and (38). Hence, from Case B in Lemma 8.7 it follows that
[TABLE]
The conclusion is hence obtained by upper bounding the right hand size of (44) using the union bound of (46) and (47). \halmos
Lemma 8.12
Let . Given , Algorithm 3 guarantees that
[TABLE]
and
[TABLE]
Proof 8.13
Proof. Since the proofs of (48) and (49) are very similar, we will only prove (48). Let
[TABLE]
and
[TABLE]
Define . Using this definition and those of , , and we have
[TABLE]
By (28) and line 5 of Algorithm 2, we have . Let , and for . From Lemma 8.6 and (33) it follows that
[TABLE]
Writing and , these two inequalities imply
[TABLE]
Hence,
[TABLE]
[TABLE]
We next find a probabilistic bound for the right hand side of the above inequality.
Bounds on and :* The inequality (33) indicates that*
[TABLE]
Define and . Since is independent of and , we have . Furthermore,
[TABLE]
where the second inequality follows from (53). Using the definition of , (54), and (31)-(32), it follows that . Hence Case A in Lemma 8.7 and union bound we get
[TABLE]
With a similar argument, we can also show
[TABLE]
Bound on :* Define and . Using (31)-(32), it is easy to verify that . Hence, from Case B in Lemma 8.7 we have*
[TABLE]
Bound on :* From definition of and (10), it follows that*
[TABLE]
Hence by Case A in Lemma 8.7 and union bound we get
[TABLE]
The conclusion can be then obtained by upper bounding the right hand side of (52) using the union bound of (55), (56), (57), and (58). \halmos
Proof 8.14
Proof of Proposition 4.1:* (i) The definition of in (12) guarantees . Recall that . With , it is straightforward to verify the following inequalities:*
[TABLE]
[TABLE]
[TABLE]
[TABLE]
Applying these four inequalities to bound the terms in (39), we get
[TABLE]
Given , let . When , we have and is monotonically decreasing in . Hence
[TABLE]
*Using the above inequality in (63) we get which completes the proof.
(ii) We only prove this corollary for the lower bounds as the proof of upper bounds is similar. The choice of guarantees . Recall that . Since , the inequalities (59), (60), (61), and (62) hold. Applying these four inequalities to (48) yields
[TABLE]
where the first inequality follows from by (11).
Let . When , the inequality (64) holds which can be applied to (65) to show that
[TABLE]
\halmos
Proof 8.15
Proof of Theorem 4.2.* We begin by establishing that the following inequalities hold with high probability in at most number of iterations:*
[TABLE]
Given , parts (i) and (ii) of Proposition 4.1 imply that when
[TABLE]
we have , , and \text{Prob}\Big{\{}\Big{|}\hat{u}^{(t)}_{*}-u^{(t)}_{*}\Big{|}>\dfrac{\epsilon_{\mathcal{A}}}{2}\Big{\}}\leq\delta/3. Hence, using union bounds we get
[TABLE]
To complete the proof we show that (66) implies and . First note that we have
[TABLE]
where the first inequality follows from (13), the second from (66), and the third holds since is a lower bound on . Using (66) and , we get
[TABLE]
In addition,
[TABLE]
where the first inequality holds by (66) and the second since is an upper bound on . The inequalities (67)-(69) complete the proof. \halmos
Proof 8.16
Proof of Corollary 5.1:* The proof directly follows from theorems 2.5 and 4.2 and definition of . \halmos*
Lemma 8.17 below shows the number of iterations required by Algorithm 4 to find the upper bound on .
Lemma 8.17
Given an input tuple , Algorithm 4 terminates with probability of at least after at most
[TABLE]
OVSMD calls and
[TABLE]
gradient iterations. In addition, and hold at termination.
Note that we use the complexity notation, which omits logarithmic terms, to simplify the expression for the gradient iteration complexity.
Proof 8.18
Proof. We first prove that Algorithm 4 terminates with a probability of at least . Consider the th iteration of this algorithm. Given returned by OVSMD, Theorem 4.2 guarantees with a probability of at least that
[TABLE]
Since , using union bound it is clear that (70) holds for with a probability of at least . In addition, (70) implies that when . Furthermore, when (which also indicates that since and ), we have
[TABLE]
where the first inequality follows from the inequality and the fact that the function is a decreasing function in . (71) indicates that as soon as , the stopping criteria of Algorithm 4 hold and the algorithm terminates with a probability of . Since and , the inequality can be guaranteed in at most iterations. Furthermore, the inequalities (70) and (71) imply that at terminatopm and .
We next compute the total number of gradient iterations taken by Algorithm 4. Notice that by Theorem 4.2, the -th call of OVSMD requires at most iterations. Therefore, the total number of iterations can be computed as
[TABLE]
where we used and in the second inequality, in the third and fourth equations, and . \halmos
Proof 8.19
Proof of Theorem 5.2:* The proof of this theorem is a direct result of Corollary 5.1 and Lemma 8.17. In particular, it is straightforward to see that the total number of OVSMD calls is*
[TABLE]
In addition, the total number of gradient iterations can be computed as
[TABLE]
\halmos
Proof 8.20
Proof of Corollary 5.3:* Let for as defined in SFLS. With a little abuse of notation, we use to represent a quantity whose order of magnitude is at least . According to Theorem 4.2, for any and , there exists satisfying such that OVSMD is a valid stochastic oracle for iteration of SFLS. Let such that . Hence, there exists such that . With such and , according to Theorem 2.5, SFLS generates a feasible solution at iteration and finds a relative -optimal and feasible solution with with a probability of at least in at most outer iterations (calls of OVSMD), which corresponds to gradient iterations. \halmos*
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Abdelaziz (2012) Abdelaziz FB (2012) Solution approaches for the multiobjective stochastic programming. European Journal of Operational Research 216(1):1–16.
- 2Abdelaziz et al. (2007) Abdelaziz FB, Aouni B, El Fayedh R (2007) Multi-objective stochastic programming for portfolio selection. European Journal of Operational Research 177(3):1811–1823.
- 3Adelman and Mersereau (2008) Adelman D, Mersereau A (2008) Relaxations of weakly coupled stochastic dynamic programs. Operations Research 56(3):712–727.
- 4Adelman and Mersereau (2013) Adelman D, Mersereau AJ (2013) Dynamic capacity allocation to customers who remember past service. Management Science 59(3):592–612.
- 5Allen-Zhu (2017) Allen-Zhu Z (2017) Katyusha: The first direct acceleration of stochastic gradient methods. Proceedings of the 49th Annual ACM Symposium on Theory of Computing , STOC ’17.
- 6Aravkin et al. (2019) Aravkin AY, Burke JV, Drusvyatskiy D, Friedlander MP, Roy S (2019) Level-set methods for convex optimization. Mathematical Programming 174(1-2):359–390.
- 7Azaron et al. (2008) Azaron A, Brown K, Tarim S, Modarres M (2008) A multi-objective stochastic programming approach for supply chain design considering risk. International Journal of Production Economics 116(1):129–138.
- 8Bach and Moulines (2013) Bach FR, Moulines E (2013) Non-strongly-convex smooth stochastic approximation with convergence rate o(1/n). Advances in Neural Information Processing Systems (NIPS) , 773–781.