Optimal Methods for Convex Risk Averse Distributed Optimization

TL;DR

This paper introduces two distributed algorithms for convex risk-averse optimization that achieve optimal or near-optimal communication complexity, addressing a key gap in the literature and demonstrating promising empirical results.

Contribution

The paper proposes the DRAO and DRAO-S algorithms, achieving optimal and near-optimal communication complexities for risk-averse distributed optimization, with the latter removing strong assumptions.

Findings

DRAO achieves optimal communication complexity under certain conditions.

DRAO-S removes assumptions and performs optimally in projection operations.

Numerical experiments show DRAO-S has encouraging empirical performance.

Abstract

This paper studies the communication complexity of convex risk-averse optimization over a network. The problem generalizes the well-studied risk-neutral finite-sum distributed optimization problem and its importance stems from the need to handle risk in an uncertain environment. For algorithms in the literature, there exists a gap in communication complexities for solving risk-averse and risk-neutral problems. We propose two distributed algorithms, namely the distributed risk averse optimization (DRAO) method and the distributed risk averse optimization with sliding (DRAO-S) method, to close the gap. Specifically, the DRAO method achieves the optimal communication complexity by assuming a certain saddle point subproblem can be easily solved in the server node. The DRAO-S method removes the strong assumption by introducing a novel saddle point sliding subroutine which only requires the projection over the ambiguity set $P$. We observe that the number of $P$-projections performed by DRAO-S is optimal. Moreover, we develop matching lower complexity bounds to show the communication complexities of both DRAO and DRAO-S to be improvable. Numerical experiments are conducted to demonstrate the encouraging empirical performance of the DRAO-S method.