Randomized Block Proximal Methods for Distributed Stochastic Big-Data Optimization

TL;DR

This paper introduces a new class of distributed algorithms for large-scale stochastic convex optimization over directed graphs, capable of handling high-dimensional, nonsmooth problems with proven convergence and tested on real datasets.

Contribution

The paper proposes a novel distributed block proximal method with convergence guarantees for stochastic big-data convex optimization over directed graphs.

Findings

Convergence to optimal cost in expectation with diminishing stepsizes.

Sublinear convergence rate with explicit bounds for constant stepsizes.

Successful application to high-dimensional classification problems on synthetic and real data.

Abstract

In this paper we introduce a class of novel distributed algorithms for solving stochastic big-data convex optimization problems over directed graphs. In the addressed set-up, the dimension of the decision variable can be extremely high and the objective function can be nonsmooth. The general algorithm consists of two main steps: a consensus step and an update on a single block of the optimization variable, which is then broadcast to neighbors. Three special instances of the proposed method, involving particular problem structures, are then presented. In the general case, the convergence of a dynamic consensus algorithm over random row stochastic matrices is shown. Then, the convergence of the proposed algorithm to the optimal cost is proven in expected value. Exact convergence is achieved when using diminishing (local) stepsizes, while approximate convergence is attained when constant stepsizes are employed. The convergence rate is shown to be sublinear and an explicit rate is provided in the case of constant stepsizes. Finally, the algorithm is tested on a distributed classification problem, first on synthetic data and, then, on a real, high-dimensional, text dataset.

Figures

Equations

$${\mathrm{minimize}}_{x\in X}$$

$$d(x)\triangleq \underset{1\le \ell \le n}{\max }{x}_{\ell }-\underset{1\le \ell \le n}{\min }{x}_{\ell }.$$

$$[A_{\delta }{]}_{ij}=\{\begin{array}{ll}\delta ,& \text{if }{A}_{ij}\ge \delta ,\\ 0,& \text{otherwise}.\end{array}$$

$${\mathrm{minimize}}_{x\in X}$$

$$x=[x_1^{\top },\dots ,x_B^{\top }{]}^{\top }.$$

$$X=X_1\times \cdots \times X_B,$$

$$\mathbb{E}[g_i(x;{\xi }_i)]=\text{textsl}{g}_i(x).$$

$$\mathbb{E}[\Vert g_i(x;{\xi }_i)\Vert ]\le G_i,\mathbb{E}[\Vert g_i(x;{\xi }_i){\Vert }^2]\le {\bar{G}}_i,$$

$${\nu }_{\ell }(a,b)={\omega }_{\ell }(b)-{\omega }_{\ell }(a)-⟨\nabla {\omega }_{\ell }(a),b-a⟩,$$

$${\nu }_{\ell }\left(\sum _{j=1}^N{\theta }_j{a}_j,b\right)\le \sum _{j=1}^N{\theta }_j{\nu }_{\ell }(a_j,b),\forall a_1,\dots ,a_N,b\in X_{\ell },$$

$${\text{prox}}_{\ell }(a,b,c)=\arg \underset{u\in X_{\ell }}{\min }\left\{⟨b,u⟩+\frac{1}{c}{\nu }_{\ell }(a,u)\right\}.$$

$$x^{t+1}=A^t{x}^t,\forall t,$$

$${\Phi }_A^{t,s}\triangleq \{\begin{array}{ll}{A}^t{A}^{t-1}\dots A^s,& \text{if }t>s,\\ A^t,& \text{if }t=s.\end{array}$$

$$\mathbb{E}[d(x^t{)}^p]$$

$$\le (\mu {)}^t\mathbb{E}[d(x^0{)}^p]\le M(\mu {)}^t\mathbb{E}[\Vert x^0{\Vert }^p],$$

$$x_{j,\ell }^t{\mid }_i=\{\begin{array}{ll}{x}_{j,\ell }^t,& \text{if }\ell ={\ell }_j^{t-1}\text{and}{s}_j^{t-1}=1\\ x_{j,\ell }^{t-1}{\mid }_i,& \text{otherwise}\end{array}$$

$$y_i^t=\sum _{j\in {\mathcal{N}}_{i,in}}{w}_{ij}{x}_j^t{\mid }_i$$

$$x_{i,\ell }^{t+1}=\{\begin{array}{ll}{\text{prox}}_{\ell }(y_{i,\ell }^t,g_{i,\ell }(y_i^t;{\xi }_i^t),{\alpha }_i^t),& \text{if }\ell ={\ell }_i^t\\ x_{i,\ell }^t,& \text{otherwise}\end{array}$$

$$y_i^t$$

$$x_{i,\ell }^{t+1}$$

$$[D_{\ell }^t{]}_{ij}=\{\begin{array}{ll}1,& \text{if }i=j\text{, }\ell ={\ell }_i^t\text{ and }{s}_i^t=1,\\ 0,& \text{otherwise}.\end{array}$$

$$y_i^t$$

$$x_{i,\ell }^{t+1}$$

$$z_{\ell }^{t+1}=W_{\ell }^t{z}_{\ell }^t,$$

$$x_{i,\ell }^{t+1}=\{\begin{array}{ll}{y}_{i,\ell }^t-{\alpha }_i^t{g}_{i,\ell }(y_i^t;{\xi }_i^t),& \text{if }\ell ={\ell }_i^t\\ x_{i,\ell }^t,& \text{otherwise}.\end{array}$$

$$x_{i,\ell }^{t+1}=\{\begin{array}{ll}{\text{prox}}_{\ell }(y_{i,\ell }^t,{\nabla }_{\ell }{h}_i(y_i^t;{\xi }_i^t),{\alpha }_i^t),& \text{if }\ell ={\ell }_i^t,\\ x_{i,\ell }^t,& \text{otherwise},\end{array}$$

$$h_i(x;{\xi }_i)=\sum _{\ell =1}^B{\hat{h}}_{i,\ell }(x_{\ell },{\xi }_i),$$

$$y_{i,\ell }^t$$

$$x_{i,\ell }^{t+1}$$

$$\sum _{t=0}^{\infty }{\alpha }_i^t=\infty ,\sum _{t=0}^{\infty }({\alpha }_i^t{)}^2<\infty ,$$

$$\sum _{t=0}^{\infty }{a}_M^t=\infty ,\sum _{t=0}^{\infty }(a_M^t{)}^2<\infty ,a_M^{t+1}\le a_M^t,$$

$$\sum _{t=0}^{\infty }{a}_m^t=\infty ,\sum _{t=0}^{\infty }(a_m^t{)}^2<\infty ,a_m^{t+1}\le a_m^t.$$

Full text

In this paper we introduce a class of novel distributed algorithms for solving stochastic big-data convex optimization problems over directed graphs. In the addressed set-up, the dimension of the decision variable can be extremely high and the objective function can be nonsmooth. The general algorithm consists of two main steps: a consensus step and an update on a single block of the optimization variable, which is then broadcast to neighbors. Three special instances of the proposed method, involving particular problem structures, are then presented. In the general case, the convergence of a dynamic consensus algorithm over random row stochastic matrices is shown. Then, the convergence of the proposed algorithm to the optimal cost is proven in expected value. Exact convergence is achieved when using diminishing (local) stepsizes, while approximate convergence is attained when constant stepsizes are employed. The convergence rate is shown to be sublinear and an explicit rate is provided in the case of constant stepsizes. Finally, the algorithm is tested on a distributed classification problem, first on synthetic data and, then, on a real, high-dimensional, text dataset.

1 Introduction

Recent years have witnessed a steadily growing interest in distributed learning and control over networks consisting of multiple smart agents. Several problems arising in this scenario can be formulated as distributed optimization problems which need to be solved by networks of agents. In this paper, we focus on the following stochastic big-data convex optimization problem, which is to be solved over a network of $N$ interconnected agents,

[TABLE]

where $X\subseteq\mathbb{R}^{n}$ is a convex set, $\xi_{i}\in\mathbb{R}$ is a random variable and the functions $h_{i}:\mathbb{R}^{n}\to\mathbb{R}$ are continuous, convex and possibly non smooth. The optimization variable $x$ is extremely high dimensional and with block structure, i.e., $n=\sum_{\ell=1}^{B}n_{\ell}$ with $n_{\ell}$ being the dimension of the $\ell$-th block and $B\gg 1$ the number of blocks. Regarding the role of stochastic functions in the considered set-up, it is worth stressing that they allow agents to deal with various type of problems. Among the others, the case of learning problems involving massive datasets is of particular interest. In this case, the local objective function typically has the form $f_{i}(x)=\frac{1}{m_{i}}\sum_{r=1}^{m_{i}}h_{i}(x,\xi_{i}^{r})$, where $\xi_{i}^{r}$, $r=1,\dots,m_{i}$, are samples uniformly drawn from a certain dataset consisting of $m_{i}$ elements. When $m_{i}$ is very large it could be computationally infeasible to compute a subgradient of the entire $f_{i}$. On the other side, given $\xi_{i}^{r}$, computing a subgradient of $h_{i}(x,\xi_{i}^{r})$ is much simpler. Problems of this type are often referred to as sample average approximation problems [2]. Other relevant classes of problems include those of dynamic, or online, optimization problems in which samples generating functions $h_{i}$ are processed as they become available [3, 4] and settings in which only noisy subgradients of the objective functions are available [5].

Applying classical distributed algorithms to big-data problems may be infeasible due, e.g., to limitations in the communication bandwidth. In fact, they would require agents to communicate a prohibitive amount of data due to the high dimension of the decision variable. This calls for tailored distributed algorithms for big-data optimization problems in which only few blocks of the entire (local) solution estimate are sent to neighbors. Thus, the literature relevant to this paper can be divided in three main (partially overlapping) categories: stochastic optimization methods, block coordinate algorithms and primal distributed algorithms.

Stochastic optimization algorithms: To the best of our knowledge, the first work dealing with stochastic problems has been [6]. Since this seminal work, there has been a steady increase in the interest for this type of problems, and algorithms for solving them (see, e.g., [7] and references therein). Among the others, stochastic approximation approaches were presented in [8, 9] and stochastic mirror descent algorithms have been studied in [10, 11]. Stochastic gradient descent algorithms are particularly appealing in learning problems (see, e.g., [12]) in which extremely large datasets are involved, since they allow for batch processing of the data.

Block coordinate algorithms: Centralized block coordinate methods have a long history (see, e.g., [13] for a survey). They were firstly designed for solving smooth problems, but, in the last years, an increasing number of results have been provided to deal with nonsmooth objective functions. Two main rules for selecting the block to be updated have been studied: cyclic (or almost cyclic; see, e.g., [14]) or random. In the last case, randomized block coordinate algorithms have been proposed [15, 16, 17, 18, 19]. Particularly relevant for this paper is the work in [11], in which a stochastic block mirror descent method with random block updates is proposed. Parallel block coordinate methods are also a well established strand of optimization literature, see, e.g., [20]. The work in [21] applies to smooth convex functions, while the ones in [22, 23, 24] face up composite optimization problems. A unified framework for nonsmooth optimization using block algorithms has been studied in [25] for centralized and parallel set-ups.

Distributed algorithms: Many distributed optimization algorithms have been proposed in recent years. In [26] a distributed gradient descent algorithm was firstly introduced, which is capable to deal with both deterministic and stochastic convex optimization problems. When the problems to be solved involve nonsmooth objective functions, subgradient-based algorithms have been designed. First examples of such algorithms appeared in [27, 28, 29], while recent advances involve more sophisticated protocols, to deal with directed communication [30, 31, 32, 33, 34]. Many distributed algorithms involving proximal operations have also been proposed (see, e.g.,[35] for a survey on proximal algorithms). Among the others, a proximal gradient method was developed in [36] to deal with unconstrained problems, while in [37, 38] proximal algorithms have been presented to deal with constrained optimization. The stochastic setting has also been treated [5, 39, 40, 41, 42, 43, 44]. In particular, a stochastic subgradient projection algorithm appeared in [5], while a stochastic distributed mirror descent was proposed in [44]. Distributed algorithms over random networks are also relevant to this paper. In [45], consensus protocols were studied using random row-stochastic matrices, while in [46] a distributed subgradient method over random networks with underlying doubly stochastic matrices has been proposed. Distributed algorithms dealing with block communication have started to appear only recently. A block gradient tracking scheme has been presented in [47] for nonconvex problems with nonsmooth regularizers, while [48] proposes an asynchronous algorithm for nonconvex optimization based on the method of multipliers, which is implementable block-wise. A randomized block-coordinate algorithm for smooth problems with common cost function and linear constraints has been presented in [49].

In this paper, we introduce the Distributed Block Proximal Method, which models a class of distributed proximal algorithms, with block communication, for solving stochastic big-data convex optimization problems with nonsmooth objective function. The communication network is modeled as a directed graph admitting a doubly stochastic weight matrix. At each iteration, each node is awake with a certain probability (and idle otherwise). If awake, it performs a consensus step, computes a stochastic subgradient of a local objective function, and performs a proximal-based update (depending on the computed subgradient and on a local stepsize) on a randomly chosen block only. Then, it exchanges with its neighbors only the updated block of the decision variable, thus requiring a small amount of communication bandwidth. We also present three special instances of the proposed algorithm. In the first one, the proximal mapping is based on the squared 2-norm, thus leading to explicit block subgradient steps. In the other two, smooth objective functions and separable (possibly nonsmooth) ones are considered. In both these cases the computational load at each node in the network can be further reduced with respect to the general algorithm. We point out that no global parameter is required in the evolution of the algorithms. In fact, each node is awake and selects blocks with locally defined probabilities, and uses local stepsizes. The block-wise updates and the communication of a single block induce nontrivial technical challenges in the algorithm analysis. On this regard, it is worth noting that, despite the double stochasticity of the weight matrix, the consensus step on each block turns out to be performed using a sequence of random row-stochastic matrices. The analysis for the Distributed Block Proximal Method is carried out in two parts. First, the convergence properties of a dynamic block consensus protocol over random graphs are studied, by building on block-wise, perturbed consensus dynamics with random matrices. A bound on the expected distance from consensus is provided, which is then specialized to the cases of constant and diminishing stepsizes respectively. Then, a bound on the expected distance from the (globally) optimal cost is provided by properly bounding errors due to the block-wise update and exploiting the probability of drawing blocks. When constant stepsizes are used, approximate convergence (with a constant error term) to the optimal cost is proven in expected value, while asymptotic exact convergence is reached for diminishing stepsizes. Finally, we provide an explicit convergence rate for the proposed algorithm when using constant stepsizes. The rate is sublinear, even though a linear term is present, which can be predominant in the first iterations.

The paper is organized as follows. The problem set-up is introduced in Section 2 along with some preliminary results. In Section 3, the Distributed Block Proximal Method is presented and three special algorithm instances are given in Section 4. Then, the algorithm is analyzed in Section 5. Finally, a numerical example involving a distributed classification problem over a syntetic and a real, high-dimensional, text document datasets is dispensed in Section 6 and some conclusions are drawn in Section 7.

2 Set-up and preliminaries

2.1 Notation and definitions

Given a vector $x\in\mathbb{R}^{n}$, we denote by $x_{\ell}$ the $\ell$-th block of $x$, i.e., given a partition of the identity matrix $I=[U_{1},\dots,U_{B}]$, with $U_{\ell}\in\mathbb{R}^{n\times n_{\ell}}$ for all $\ell$ and $\sum_{\ell=1}^{B}n_{\ell}=n$, it holds $x=\sum_{\ell=1}^{B}U_{\ell}x_{\ell}$ and $x_{\ell}=(U_{\ell})^{\top}x$. Moreover we denote by $\|x\|$ the 2-norm of $x$. Given a vector $x\in\mathbb{R}^{n}$, with scalar blocks, we define

[TABLE]

Given a vector $x_{i}\in\mathbb{R}^{n}$, we denote by $x_{i,\ell}$ the $\ell$-th block of $x_{i}$. Moreover, given a constant $c$, and an index $t$, we denote by $(c)^{t}$, $c$ to the power of $t$, while given a sequence $\{x^{t}\}_{t\geq 0}$, we denote by $x^{t}$ the $t$-th element of the sequence. Given a matrix $A$, we denote by $a_{ij}$ (or $[A]_{ij}$) the element of $A$ located at row $i$ and column $j$. Given two matrices $A$ and $B$, we write $A\geq B$ if $a_{ij}\geq b_{ij}$ for all $i$ and $j$. Given two vectors $a,b\in\mathbb{R}^{n}$ we denote by $\langle a,b\rangle$ their scalar product. Given a discrete random variable $r\in\{1,\ldots,R\}$, we denote by $P(r=\bar{r})$ the probability of $r$ to be equal to $\bar{r}$. Given a nonsmooth function $f$, we denote by $\partial f(x)$ its subdifferential computed at $x$, and by $\partial_{x_{\ell}}f(x)$ the subdifferential of $f$ with respect to the $\ell$-th block of $x$.

We say that a directed graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$ contains a spanning tree if for some $v\in\mathcal{V}$ there exists a directed path from the vertex $v$ to all other vertices $u\in\mathcal{V}$. Given a nonnegative matrix $A$ and some $\delta\in(0,1)$, we denote by $A_{\delta}$ the matrix whose entries are defined as

[TABLE]

We say that $A$ contains a $\delta$-spanning tree if the graph induced by $A_{\delta}$ contains a spanning tree.

2.2 Distributed stochastic optimization set-up

As anticipated in the introduction, we consider the following optimization problem,

[TABLE]

We recall that $\xi_{i}$ is a random variable, functions $h_{i}:\mathbb{R}^{n}\to\mathbb{R}$ are continuous, convex and possibly nonsmooth for every $\xi_{i}$, $X\subseteq\mathbb{R}^{n}$ and $n\gg 1$. We let $f_{i}(x)=\mathbb{E}[h_{i}(x;\xi_{i})]$ and $f(x)=\sum_{i=1}^{N}f_{i}(x)$. Moreover, $x^{\star}\in\mathbb{R}^{n}$ is a solution of problem (1). The optimization variable $x\in\mathbb{R}^{n}$ has a block structure, i.e.,

[TABLE]

with $x_{\ell}\in\mathbb{R}^{n_{\ell}}$ for all $\ell$ and $\sum_{\ell=1}^{B}n_{\ell}=n$. We make the following assumption on the problem structure

Assumption 1** (Problem structure).**

1. (A)

The constraint set $X$ has the block structure

[TABLE]

where, for $\ell=1,\dots,B$, the set $X_{\ell}\subseteq\mathbb{R}^{n_{\ell}}$ is closed and convex, and $\sum_{\ell=1}^{B}n_{\ell}=n$. 2. (B)

Let $g_{i}(x;\xi_{i})\in\partial h_{i}(x;\xi_{i})$ (resp. $\textsl{g}_{i}(x)\in\partial f_{i}(x)$) be a subgradient of $h_{i}(x;\xi_{i})$ (resp. $f_{i}(x)$) computed at $x$. Then, $g_{i}(x;\xi_{i})$ is an unbiased estimator of the subgradient of $f_{i}$, i.e.,

[TABLE] 3. (C)

There exist constants $G_{i}\in[0,\infty)$ and $\bar{G}_{i}\in[0,\infty)$ such that

[TABLE]

for all $x$ and $\xi_{i}$, for all $i\in\{1,\ldots,N\}$. $\square$

Notice that, if $X=\mathbb{R}^{n}$, Assumption 1(A) is clearly satisfied. Moreover, let us denote by $g_{i,\ell}(x;\xi_{i})$ the $\ell$-th block of $g_{i}(x;\xi_{i})$ and let $\textsl{g}(x)\in\partial f(x)$ be a subgradient of $f$ computed at $x$. Then, Assumption 1(C) implies that $\mathbb{E}[\|g_{i,\ell}(x;\xi_{i})\|]\leq G_{i}$ for all $\ell\in\{1,\ldots,B\}$ and $\|\textsl{g}_{i}(x)\|\leq G_{i}$. Moreover, let $\bar{G}\triangleq\sum_{i=1}^{N}\bar{G}_{i}$ and $G\triangleq\sum_{i=1}^{N}G_{i}$. Then, $\|\textsl{g}(x)\|\leq G$ and $\|\textsl{g}_{i}(x)\|\leq G$ for all $i$.

Problem (1) is to be solved in a distributed way by a network of $N$ agents. Each agent in the network is assumed to know only a portion of the entire problem, namely agent $i$ knows $f_{i}$ and the constraint set $X$ only. We make the following assumption on the network structure.

Assumption 2** (Communication structure).**

1. (A)

The network is modeled through a weighted strongly connected directed graph $\mathcal{G}=(\mathcal{V},\mathcal{E},{W})$ with $\mathcal{V}=\{1,\ldots,N\}$, $\mathcal{E}\subseteq\mathcal{V}\times\mathcal{V}$ and ${W}\in\mathbb{R}^{N\times N}$ being the weighted adjacency matrix. We denote by $\mathcal{N}_{i,out}$ the set of out-neighbors of node $i$, i.e., $\mathcal{N}_{i,out}\triangleq\{j\mid(i,j)\in\mathcal{E}\}\cup\{i\}$. Similarly, the set of in-neighbors of node $i$ is defined as $\mathcal{N}_{i,in}\triangleq\{j\mid(j,i)\in\mathcal{E}\}\cup\{i\}$. 2. (B)

For all $i,j\in\{1,\ldots,N\}$, the weights $w_{ij}$ of the weight matrix ${W}$ satisfy

1. (i)

if $i\neq j$, $w_{ij}>0$ if and only if $j\in\mathcal{N}_{i,in}$; 2. (ii)

there exists a constant $\eta>0$ such that $w_{ii}\geq\eta$ and if $w_{ij}>0$, then $w_{ij}\geq\eta$; 3. (iii)

$\sum_{j=1}^{N}w_{ij}=1$ and $\sum_{i=1}^{N}w_{ij}=1$. $\square$

A function $\omega_{\ell}$ is associated to the $\ell$-th block of the optimization variable for all $\ell$. Let the function $\omega_{\ell}:X_{\ell}\to\mathbb{R}$, be continuously differentiable and $\sigma_{\ell}$-strongly convex. Functions $\omega_{\ell}$ are sometimes referred to as distance generating functions. Then, we define the proximal function, also called Bregman’s divergence, associated to $\omega_{\ell}$ as

[TABLE]

for all $a,b\in X_{\ell}$. The following assumption is made on the functions $\nu_{\ell}$.

Assumption 3** (Bregman’s divergence separate convexity).**

For all $\ell\in\{1,\ldots,B\}$, the function $\nu_{\ell}$ satisfies

[TABLE]

where $\sum_{j=1}^{N}\theta_{j}=1$ and $\theta_{j}\geq 0$ for all $j$. $\square$

Notice that the above assumption is satisfied by many functions (such as the quadratic function, the Boltzmann-Shannon entropy and the exponential function) and conditions on the functions $\omega_{\ell}$ guaranteeing (2) can be provided (see [50]). Finally, given $a\in X_{\ell}$, $b\in\mathbb{R}^{n_{\ell}}$ and $c\in\mathbb{R}$, the proximal mapping associated to $\nu_{\ell}$ is defined as

[TABLE]

2.3 Preliminary results

Consider a stochastic, discrete-time dynamical system evolving according to

[TABLE]

where $\{A^{t}\}_{t\geq 0}$ is a sequence of random $n\times n$ row-stochastic matrices. Let $(\Omega,\mathcal{F},P)$ be a probability space. We assume that the sequence $\{A^{t},\mathcal{S}^{t}\}_{t\geq 0}$ forms an adapted process, i.e., $\{A^{t}\}_{t\geq 0}$ is a stochastic process defined on $(\Omega,\mathcal{F},P)$, $\{\mathcal{S}^{t}\}_{t\geq 0}$ is a filtration (i.e., $\mathcal{S}^{t}\subseteq\mathcal{S}^{t+1}$ and $\mathcal{S}^{t}\subseteq\mathcal{F}$ for all $t$) and $A^{t}$ is measurable with respect to $\mathcal{S}^{t}$. Given a sequence of matrices $\{A^{t}\}_{t\geq 0}$, let us define the transition matrix from iteration $s$ to iteration $t$ as

[TABLE]

Then, the following result, adapted from [45, Theorem 3.1], holds true for system (4).

Lemma 1** ([45, Theorem 3.1]).**

Consider system (4). If there exist $h>0$, $\delta>0$ such that $\mathbb{E}[\sum_{t=mh+1}^{(m+1)h}A^{t}\mid\mathcal{S}^{mh}]$ contains a $\delta$-spanning tree for each $m$, and $A^{t}\geq\delta I$ for each $t$, then, for any given initial distribution of $x^{0}$ with $\mathbb{E}[\|x^{0}\|^{p}]<\infty$ (which is independent of $\{A^{t}\}_{t\geq 0}$), and any $p>0$, it holds

[TABLE]

where $M\in(0,\infty)$ and $\mu\in(0,1)$. $\square$

Finally, the following three results will be useful in the rest of the paper.

Lemma 2**.**

Given a scalar $\beta\neq 1$, it holds that

1. (i)

for any $t\geq r\geq 0$, $\sum_{s=r}^{t}(\beta)^{s}=\frac{(\beta)^{r}-(\beta)^{t+1}}{1-\beta}$ 2. (ii)

*for any $t\geq 0$, $\sum_{s=0}^{t}\sum_{\tau=0}^{t-s}(\beta)^{\tau}=\frac{t+1-\beta(t+2)+(\beta)^{t+2}}{(1-\beta)^{2}}$ *$\square$

Lemma 3** ([5, Lemma 3.1]).**

Let $\{\gamma^{t}\}_{t\geq 0}$ be a scalar sequence.

1. (i)

If $\lim_{t\to\infty}\gamma^{t}=\gamma$ and $\beta\in(0,1)$ then $\lim_{t\to\infty}\sum_{s=0}^{t}(\beta)^{t-s}\gamma^{s}=\frac{\gamma}{1-\beta}$. 2. (ii)

*If $\gamma^{t}\geq 0$ $\forall t$, $\sum_{t=0}^{\infty}\gamma^{t}<\infty$ and $\beta\in(0,1)$, then $\sum_{t=0}^{\infty}\left(\sum_{s=0}^{t}(\beta)^{t-s}\gamma^{s}\right)<\infty$. *$\square$

Lemma 4** (Tower property of conditional expectation).**

Let $X$ be a random variable defined on a probability space $(\Omega,\mathcal{F},P)$. Let $\mathcal{Z}\subseteq\mathcal{Y}\subseteq\mathcal{F}$. Then, $\mathbb{E}[\mathbb{E}[X\mid\mathcal{Y},\mathcal{Z}]\mid\mathcal{Z}]=\mathbb{E}[X\mid\mathcal{Z}].$ $\square$

3 Distributed Block Proximal Method

The Distributed Block Proximal Method for solving problem (1) in a distributed way is now introduced. The algorithm works as follows. Each agent $i$ maintains a local solution estimate $x_{i}^{t}$ and a local copy of the estimates of its in-neighbors. Let us denote by $x_{j}^{t}{\mid}_{i}$ the copy of the solution estimate of agent $j$ at agent $i$. At the beginning, each node initializes its state with a random (bounded) initial condition $x_{i}^{0}$ which is then shared with its neighbors. At each iteration each agent $i$ is awake with probability $p_{i,on}\in(0,1]$ and idle with probability $1-p_{i,on}$. Thus, the proposed algorithm models a particular type of asynchrony in which the communication graph is fixed and agents can communicate or not with their neighbors with a certain probability. If agent $i$ is awake, it picks randomly a block $\ell_{i}^{t}\in\{1,\ldots,B\}$, some $\xi_{i}^{t}$, and performs two updates:

1. (i)

it computes a weighted average of its in-neighbors’ estimates $x_{j}^{t}{\mid}_{i}$, $j\in\mathcal{N}_{i,in}$; 2. (ii)

it computes $x_{i}^{t+1}$ by updating the $\ell_{i}^{t}$-th block of $x_{i}^{t}$ through a proximal mapping step and leaving the other blocks unchanged.

Then, it broadcasts $x_{i,\ell_{i}^{t}}^{t+1}$ to its out-neighbors. We model the status (awake or idle) of each node $i$ at each iteration $t$ through a random variable $s_{i}^{t}\in\{0,1\}$ which is $1$ (corresponding to being awake) with probability $p_{i,on}$ and [math] with probability $1-p_{i,on}$. A pseudocode of the method is reported in Algorithm 1.

Notice that all the quantities involved in the above algorithm are local for each node. In fact, each node has locally defined probabilities (both of awakening and block drawing) and local stepsizes.

Moreover, it is worth noting that, despite node $i$ receives from each $j\in\mathcal{N}_{i}^{in}$ only the block $x_{j,\ell_{j}^{t-1}}^{t}$, the consensus step (6) is in fact performed by using the entire $x_{j}^{t}$. Indeed, the other blocks have not changed since the last time they have been received. This is formalized in the next result.

Lemma 5**.**

Let Assumption 2 hold. Then $x_{j}^{t}{\mid}_{i}=x_{j}^{t}$ for all $t$. Moreover, Algorithm 1 can be compactly rewritten as follows. For all $i\in\{1,\ldots,N\}$ and all $t$, if $s_{i}^{t}=1$,

[TABLE]

else, $x_{i}^{t+1}=x_{i}^{t}$.

Proof.

The fact that $x_{j}^{t}{\mid}_{i}=x_{j}^{t}$ for all $i$ and all $t$ follows immediately from the evolution of the algorithm. In fact, the received block $x_{j,\ell_{j}^{t-1}}^{t}{\mid}_{i}$ is the only block that node $j$ has modified in the last iteration, while the others have remained unchanged. Hence, since the graph $\mathcal{G}$ is fixed, it is clear that $x_{j}^{t}{\mid}_{i}=x_{j}^{t}$ for all $i$ and all $t$. The reformulation of Algorithm 1 as (8)-(9) is then immediate from Assumption 2(B). ∎

In virtue of the previous result, in order to lighten the notation in the subsequent analysis, we will use (8)-(9) in place of Algorithm 1, by making the block communication implicit.

As for the block-wise proximal update (9), the $\ell_{i}^{t}$-th block of a whole stochastic subgradient computed at $y_{i}^{t}$ is used. Unfortunately, computing a subgradient with respect to the $\ell_{i}^{t}$-th component only is, in general, not equivalent to picking the $\ell_{i}^{t}$-th block of a whole subgradient $g_{i}(y_{i}^{t};\xi_{i}^{t})$. In fact, in general it holds that, picking $g_{1}\in\partial_{y_{i,1}}h_{i}(y_{i}^{t};\xi_{i}^{t})$,…, $g_{B}\in\partial_{y_{i,B}}h_{i}(y_{i}^{t};\xi_{i}^{t})$ does not imply $[g_{i}^{\top},\dots,g_{B}^{\top}]^{\top}\in\partial h_{i}(y_{i}^{t};\xi_{i}^{t})$. This will turn out to be extremely important in the subsequent analysis. If functions $f_{i}$ are separable on the blocks, then, only the subgradient with respect to the $\ell_{i}^{t}$-th component can be computed. Similarly, if the functions $f_{i}$ are smooth, the $\ell_{i}^{t}$-th block of the gradient can be directly computed as the gradient with respect to that block. In these cases, the computational load at each node can be further reduced, as it will be shown in Section 4.

The last key feature of the Distributed Block Proximal Method involves the consensus step (8). Let $z_{\ell}^{t}$ be the vector stacking the $\ell$-th component of all the $x_{i}^{t}$, i.e., $z_{\ell}^{t}\triangleq[(x_{1,\ell}^{t})^{\top},\dots,(x_{N,\ell}^{t})^{\top}]^{\top}$. Also, let ${D}_{\ell}^{t}$ be a diagonal matrix in which the $i$-th element of the diagonal is set to $1$ if $s_{i}^{t}=1$ and $\ell_{i}^{t}=\ell$, and it is set to [math] otherwise, i.e.,

[TABLE]

Finally, let ${D}_{-\ell}^{t}=I-{D}_{\ell}^{t}$. Now, consider a consensus protocol associated to the Distributed Block Proximal Method, i.e.,

[TABLE]

This system can be rewritten in terms of $z_{\ell}$ as

[TABLE]

where ${W}_{\ell}^{t}\triangleq{D}_{-\ell}^{t}+{D}_{\ell}^{t}{W}$. It can be easily verified that, for all $\ell$ and $t$, the matrix ${W}_{\ell}^{t}$ is row-stochastic but not doubly stochastic anymore (unless all nodes select the same block $\ell$ at some iteration $t$).

Remark 1*.*

It is worth noting that the proposed algorithm, besides being easy to implement, considers challenges that cannot be addressed by other block-wise distributed algorithms [47, 48, 49]. In particular, none of those works deals with stochastic problems. Moreover, in [47] composite objective functions with non-smooth components are considered but the non-smooth part must be common to all the agents. In [48, 49] at least differentiability of the objective is required. Finally, in our algorithm, all the algorithm parameters are local.

4 Special instances

In this section, three special cases of the Distributed Block Proximal Method are presented. The first one is obtained by choosing the squared 2-norm as distance generating function, while the other two result from smooth and separable objective functions respectively.

4.1 Distributed Block Subgradient Method

By using $\omega_{\ell}(x)=\frac{1}{2}\|x\|^{2}$ for all $\ell$, and assuming $X=\mathbb{R}^{n}$, the proximal mapping (3) has an explicit analytical solution and the update step (7) becomes

[TABLE]

Notice that, the proximal step becomes a subgradient step on a single block of the optimization variable. Thus, we call Distributed Block Subgradient Method the resulting algorithm, i.e., the one obtained by replacing (7) with (10) in Algorithm 1. Notice that, in this case, it holds that, for all $\ell$, the strong convexity parameter is $\sigma_{\ell}=1$, thus resulting in special bounds in the subsequent algorithm analysis.

4.2 Smooth functions

The update of the solution estimate in (7) requires, in general, for node $i$ at iteration $t$, the computation of an entire stochastic subgradient at the point $y_{i}^{t}$. However, only the $\ell_{i}^{t}$-th block of the computed subgradient is used in the update step. When a function $h_{i}$ is smooth, however, the $\ell_{i}^{t}$-th block of its gradient can be directly computed as the gradient of $h_{i}$ with respect to the $\ell_{i}^{t}$-th block of the optimization variable and (7) can be replaced by

[TABLE]

where $\nabla_{\ell}h_{i}$ denotes the (partial) gradient of $h_{i}$ with respect to the $\ell$-th block of the optimization variable. Thus, when smooth functions are involved in the problem, the computational load can be reduced by avoiding the computation of the entire (sub)gradient.

4.3 Separable functions

When functions $h_{i}(x;\xi_{i})$ are separable, i.e.,

[TABLE]

the Distributed Block Proximal Method can be further simplified, allowing for an extra reduction of the computational load at each iteration at a given node. In fact, it holds that $\partial h_{i}(y_{i}^{t};\xi_{i}^{t})=\partial_{y_{i,1}}h_{i}(y_{i}^{t};\xi_{i}^{t})\times\dots\partial_{y_{i,B}}h_{i}(y_{i}^{t};\xi_{i}^{t})$, and hence $g_{i}(y_{i}^{t};\xi_{i}^{t})=\sum_{\ell=1}^{B}\hat{g}_{i,\ell}(y_{i,\ell}^{t},\xi_{i}^{t}),$ where $\hat{g}_{i,\ell}\in\partial\hat{h}_{i,\ell}$ is a subgradient of $\hat{h}_{i,\ell}$. This implies that $g_{i,\ell}(y_{i}^{t},\xi_{i}^{t})=\hat{g}_{i,\ell}(y_{i,\ell}^{t},\xi_{i}^{t})$ and, thus, only the $\ell_{i}^{t}$-th block of $y_{i}^{t}$ is needed in order to compute $g_{i,\ell}(y_{i}^{t},\xi_{i}^{t})$ and hence $x_{i}^{t+1}$. Thus the Distributed Block Proximal Method can be simplified by allowing nodes with a separable function to reduce their computational load. In particular, assume the cost function of node $i$ to be separable. Then, a single block of $y_{i}^{t}$ can be updated at each iteration and a subgradient can be directly computed for the corresponding block, without computing an entire subgradient. Hence, the algorithm can be rewritten, by using the equivalent formulation in Lemma 5, as follows. If $s_{i}^{t}=1$,

[TABLE]

else, $x_{i}^{t+1}=x_{i}^{t}$.

5 Algorithm analysis

In this section, the convergence of the Distributed Block Proximal Method is proven in expected value. The proof consists of two main parts. In the first one the consensus of the agents’ solution estimates is shown, while in the second one convergence towards the optimal cost is proven. Both results are given, at first, in a general form and, then, specialized to the case of constant stepsizes (in which convergence to a neighborhood is proven) and diminishing stepsizes (in which exact asymptotic convergence is reached).

Define $a^{t}\triangleq[\alpha_{1}^{t},\dots,\alpha_{N}^{t}]^{\top}$, $a_{M}^{t}\triangleq\max_{i}\alpha_{i}^{t}$ and $a_{m}^{t}\triangleq\min_{i}\alpha_{i}^{t}$. We summarize in the following two assumptions, the two different choices for the stepsize sequences we consider in the following analysis.

Assumption 4** (Constant stepsize).**

The sequences $\{\alpha_{i}^{t}\}_{t\geq 0}$ satisfy $\alpha_{i}^{t}=\alpha_{i}>0$ for all $t$ and all $i$.

Assumption 5** (Diminishing stepsize).**

The sequences $\{\alpha_{i}^{t}\}_{t\geq 0}$ satisfy

[TABLE]

for all $i\in\{1,\ldots,N\}$. Moreover, $\alpha_{i}^{t+1}\leq\alpha_{i}^{t}$ for all $t$ and all $i\in\{1,\ldots,N\}$.

Notice that, under Assumption 4, $a_{M}^{t}\triangleq a_{M}=\max_{i}\alpha_{i}$ and $a_{m}^{t}\triangleq a_{m}=\min_{i}\alpha_{i}$ for all $t$, while, under Assumption 5 it can be easily verified that

[TABLE]

and

[TABLE]

Define the vector stacking all local solution estimates as $\bm{x}(t)\triangleq[(x_{1}(t))^{\top},\dots,(x_{N}(t))^{\top}]^{\top}$, and the average (over the agents) of the local estimates at $t$ as

[TABLE]

Then, we make the following assumption on the random variables involved in the algorithm.

Assumption 6** (Random variables).**

1. (A)

For a given $i\in\{1,\ldots,N\}$, the random variables $\ell_{i}^{t}$ and $s_{i}^{t}$ are independent and identically distributed for all $t$. 2. (B)

For a given $t$, the random variables $s_{i}^{t}$, $\ell_{i}^{t}$ and $\xi_{i}^{t}$ are independent of each other for all $i\in\{1,\ldots,N\}$. 3. (C)

There exist constants $C_{i}\in[0,\infty)$ such that $\mathbb{E}[\|x_{i}^{0}\|]\leq C_{i}$ for all $i\in\{1,\ldots,N\}$ and hence $\mathbb{E}[\|\bm{x}^{0}\|]\leq C=\sum_{i=1}^{N}C_{i}$. $\square$

Before proceeding with the algorithm analysis, let us provide a preliminary instrumental result. Define $q_{i}^{t}=x_{i,\ell_{i}^{t}}^{t+1}-y_{i,\ell_{i}^{t}}^{t}$ and $q^{t}=[(q_{1}^{t})^{\top},\dots,(q_{N}^{t})^{\top}]^{\top}$. Then, the following result applies.

Lemma 6**.**

Let Assumptions 1(A) and 1(C) hold. Then,

[TABLE]

for all $i\in\{1,\ldots,N\}$, where $\sigma=\min_{\ell}\sigma_{\ell}$.

Proof.

The first order necessary optimality condition on (9) for $\ell=\ell_{i}^{t}$ reads

[TABLE]

for all $u\in X_{\ell_{i}^{t}}$. Notice now that, by definition, $y_{i,\ell_{i}^{t}}^{t}\in X_{\ell_{i}^{t}}$, since it is a weighted average of points lying in $X_{\ell_{i}^{t}}$. Thus, by taking $u=y_{i,\ell_{i}^{t}}^{t}$, one obtains

[TABLE]

where we have used the strong convexity of $\omega_{\ell_{i}^{t}}$. By rearranging the terms, one has

[TABLE]

and hence,

[TABLE]

Now, by taking the expected value and using the subgradient boundedness from Assumption 1(C), one gets

[TABLE]

thus concluding the proof. ∎

5.1 Dynamic consensus with random matrices

In this section we show that the sequences $\{x_{i}^{t}\}_{t\geq 0}$ and $\{y_{i}^{t}\}_{t\geq 0}$ computed by each agent in the network asymptotically achieve consensus in expected value when using diminishing stepsizes. Moreover, an upper bound on the distance from consensus is provided in the case of constant stepsizes.

Let $\mathcal{S}^{t}\triangleq\{\bm{x}^{\tau}\mid\tau\in\{0,\dots,t\}\}$ be the set of estimates generated by the Distributed Block Proximal Method up to iteration $t$ (which is indeed a filtration). Moreover, define the probability of node $i$ to both be awake and pick block $\ell$ at each iteration as

[TABLE]

Then, the following lemma provides a bound on the expected distance between $x_{i}^{t}$ and the average $\bar{x}^{t}$ (defined in (14)).

Lemma 7**.**

Let Assumptions 1(C), 2, 6 hold. Then, there exist constants $M\in(0,\infty)$ and $\mu_{M}\in(0,1)$ such that

[TABLE]

for all $i\in\{1,\ldots,N\}$ and all $t\geq 1$.

Proof.

For the sake of presentation, assume that the blocks are scalars, i.e., $B=n$. Let us recall that $z_{\ell}^{t}$ defines the vector stacking the $\ell$-th component of all the $x_{i}^{t}$, i.e., $z_{\ell}^{t}\triangleq[x_{1,\ell}^{t},\dots,x_{N,\ell}^{t}]^{\top}$, while the matrix ${D}_{\ell}^{t}\in\mathbb{R}^{N\times N}$ is a diagonal matrix in which the $i$-th element of the diagonal is set to $1$ if $s_{i}^{t}=1$ and $\ell_{i}^{t}=\ell$ and it is set to [math] otherwise, i.e.,

[TABLE]

Consistently, we let ${D}_{-\ell}^{t}=I-{D}_{\ell}^{t}$. Notice that, for all $t$, ${D}_{\ell}^{t}$ is a random matrix whose diagonal element $[{D}_{\ell}^{t}]_{ii}$ is $1$ with probability $\pi_{i,\ell}$ and [math] with probability $1-\pi_{i,\ell}$. Define

[TABLE]

Then, by using Assumption 6(A), it can be verified that

[TABLE]

and, similarly,

[TABLE]

for all $t$.

Now, Algorithm 1 can be rewritten with respecte to $z_{\ell}$ as

[TABLE]

where ${W}_{\ell}^{t}\triangleq{D}_{-\ell}^{t}+{D}_{\ell}^{t}{W}$ is, by definition, a row-stochastic matrix, and $e_{\ell}^{t}\triangleq{D}_{\ell}^{t}q^{t}$. Now, by recursively applying (21), it holds that

[TABLE]

where $\Phi_{W_{\ell}}^{t,s}$ is the transition matrix from iteration $s$ to iteration $t$ associated to the matrices ${W}_{\ell}^{\tau}$, $\tau=s,\dots,t$. Moreover, by applying the $d(\cdot)$ operator on both sides (recall that $d(z_{\ell})=\max_{1\leq i\leq N}z_{\ell,i}-\min_{1\leq i\leq N}z_{\ell,i}$), one has

[TABLE]

Notice now that, by using (19) and (20),

[TABLE]

for all $t$. It can be seen that such a matrix is row stochastic and contains a $\delta$-spanning tree with $\delta\geq(\eta\min_{i}\pi_{i,\ell}+\min_{i}(1-\pi_{i,\ell}))>0$ (since from Assumption 2 the matrix ${W}$ contains a spanning tree), where $\eta$ is defined in Assumption 2(B). Moreover, by Assumption 6(A), $\{{W}_{\ell}^{t}\}_{t\geq 0}$ is a sequence of i.i.d. random matrices with ${W}_{\ell}^{t}\geq\eta I$. Hence, from Lemma 1, by taking the expectation on both sides of (22), we get

[TABLE]

where we used the fact that, from Lemma 6,

[TABLE]

Let us now define

[TABLE]

Since $\min_{j}z_{\ell,j}^{t}\leq\bar{z}_{\ell}^{t}\leq\max_{j}z_{\ell,j}^{t}$, for all $t$, we have that

[TABLE]

for all $i\in\{1,\ldots,N\}$. Notice now that, by definition $x_{i,\ell}^{t}=z_{\ell,i}^{t}$ and $\bar{x}_{\ell}^{t}=\bar{z}_{\ell}^{t}$. Hence,

[TABLE]

Finally, since $\|x_{i}^{t}-\bar{x}^{t}\|\leq\sum_{\ell=1}^{B}|x_{i,\ell}^{t}-\bar{x}_{\ell}^{t}|$, one has

[TABLE]

where $\mu_{M}=\max_{\ell}\mu_{\ell}$. The proof is concluded by using Assumption 6(C). ∎

Moreover, the expected value of the distance between $y_{i}^{t}$ and $x_{i}^{t}$ can be bounded, by exploiting the convexity of the norm and using Lemma 7, as stated in the next result.

Lemma 8**.**

Let Assumptions 1(C), 2, 6 hold. Then,

[TABLE]

for all $i\in\{1,\ldots,N\}$ and all $t\geq 1$.

Proof.

Form the definition of $y_{i}^{t}$ and using the convexity of the norm, one has

[TABLE]

By taking the expected value on both sides and using Lemma 7, the proof follows by noting that $\sum_{j=1}^{N}w_{ij}=1$ from Assumption 2(B). ∎

5.1.1 Constant stepsize

The following two results respectively provide an upper bound on the distance of $x_{i}^{t}$ from $\bar{x}^{t}$ as $t\to\infty$ and characterize the quantity $\sum_{\tau=0}^{t}\mathbb{E}[\|x_{i}^{\tau}-\bar{x}^{\tau}\|]$ for each $t$ in the case of constant stepsizes.

Lemma 9**.**

Let Assumptions 1(C), 2, 4, 6 hold. Then, there exist constants $M\in(0,\infty)$ and $\mu_{M}\in(0,1)$ such that

[TABLE]

for all $i\in\{1,\ldots,N\}$, with

[TABLE]

Proof.

Equation (18) in Lemma 7 consists of three terms. For the first one, $\lim_{t\to\infty}(\mu_{M})^{t-1}C=0$, since $\mu_{M}<1$. For the second term $\sum_{s=0}^{t-2}(\mu_{M})^{t-s-2}a_{M}^{s}$, by Assumption 4 and Lemma 3, one has that $\lim_{t\to\infty}\sum_{s=0}^{t-2}(\mu_{M})^{t-s-2}a_{M}^{s}=\frac{a_{M}}{1-\mu_{M}}$. The proof is completed by noting that, under Assumption 4 the last term is constant. ∎

Lemma 10**.**

Let Assumptions 1(C), 2, 4, 6 hold. Then,

[TABLE]

for all $i\in\{1,\ldots,N\}$, with

[TABLE]

Proof.

By using Assumption 6(C), for $\tau=0$, one has

[TABLE]

Hence, $\sum_{\tau=0}^{t}\mathbb{E}[\|x_{i}^{\tau}-\bar{x}^{\tau}\|]\leq C+\sum_{\tau=1}^{t}\mathbb{E}[\|x_{i}^{\tau}-\bar{x}^{\tau}\|]$ and, from Lemma 7, we have

[TABLE]

where in the last line we have rearranged the summations. Now, by noting that

[TABLE]

and using Lemma 2 the result follows through straightforward manipulations. ∎

Notice that in virtue of Lemma 8, by using the same reasoning used in the previous two results it is possible to show that

[TABLE]

and

[TABLE]

These bounds will be used in the following optimality analysis.

5.1.2 Diminishing stepsize

When adopting diminishing stepsizes, asymptotic (exact) consensus can be reached as stated in the following result.

Lemma 11**.**

Let Assumptions 1(C), 2, 5, 6 hold. Then,

[TABLE]

for all $i\in\{1,\ldots,N\}$.

Proof.

The proof is based on the same arguments as in Lemma 9. By noting that $a_{M}^{t}\to 0$ as $t\to\infty$ and that, under Assumption 5, from Lemma 3, $\lim_{t\to\infty}\sum_{s=0}^{t-2}(\mu_{M})^{t-s-2}a_{M}^{s}=0$, the result follows. ∎

The next result shows that $\lim_{t\to\infty}\sum_{\tau=0}^{t}a_{m}^{\tau}\mathbb{E}[\|x_{i}^{\tau}-\bar{x}^{\tau}\|]$ is a summable series for all $i\in\{1,\ldots,N\}$. Notice that this result does not hold in the case of constant stepsizes.

Lemma 12**.**

Let Assumptions 1(C), 2, 5, 6 hold. Then,

[TABLE]

for all $i\in\{1,\ldots,N\}$.

Proof.

As for $\tau=0$, from (5.1.1) we have $a_{m}^{0}\mathbb{E}[\|x_{i}^{0}-\bar{x}^{0}\|]\leq a_{m}^{0}C$, while, for $\tau\geq 1$, from Lemma 7 it holds that

[TABLE]

Since, by Assumption 5, $a_{m}^{\tau+1}\leq a_{m}^{\tau}\leq a_{M}^{\tau}$ for all $\tau$, one has

[TABLE]

and then,

[TABLE]

Now, from Assumption 6(C), $C<\infty$. Moreover, by using Assumption 5 and Lemma 3, we have $\lim_{t\to\infty}\sum_{\tau=1}^{t}(\mu_{M})^{\tau-1}a_{m}^{\tau}<\infty$. Finally, by Assumption 5, $\sum_{\tau=0}^{\infty}(a_{M}^{\tau})^{2}<\infty$, so that, from Lemma 3, $\lim_{t\to\infty}\sum_{\tau=1}^{t}\sum_{s=0}^{\tau-2}(\mu_{M})^{\tau-s-2}(a_{M}^{s})^{2}{<}\infty$, thus concluding the proof. ∎

As in the case of constant stepsizes, thanks to Lemma 8, it can be shown that $\lim_{t\to\infty}\mathbb{E}[\|y_{i}^{t}-x_{i}^{t}\|]=0$ and $\lim_{t\to\infty}\sum_{\tau=0}^{t}a_{m}^{\tau}\mathbb{E}[\|y_{i}^{\tau}-x_{i}^{\tau}\|]<\infty$.

5.2 Optimality

In this section, we show the convergence of the Distributed Block Proximal Method. First, a bound on the expected distance from the optimal cost at iteration $t$ is given without any assumption on the stepsize sequence. Then, it is shown that such a distance goes to [math] as $t\to\infty$ for diminishing stepsizes, while it is upper bounded by a finite quantity for constant stepsizes and an explicit convergence rate is provided.

We start by defining the Ljapunov function

[TABLE]

and $V^{t}\triangleq\sum_{i=1}^{N}V_{i}^{t}$. Moreover, given a sequence of points $\{z^{\tau}\}_{\tau=0}^{t}$, we define

[TABLE]

Then, the following result holds true.

Theorem 1**.**

Let Assumptions 1, 2, 3, 6 hold. Then,

[TABLE]

Proof.

In order to simplify the notation, let us denote $\textsl{g}_{i}^{\tau}=\textsl{g}_{i}(y_{i}^{\tau})$ and $g_{i,\ell_{i}^{\tau}}=g_{i,\ell_{i}^{\tau}}(y_{i}^{\tau},\xi_{i}^{\tau})$. From the convexity of $f$ we have that, at a given iteration $t$,

[TABLE]

Now, we make some manipulation on the term $\mathbb{E}[f(\bar{x}^{\tau})]-f(x^{\star})=\mathbb{E}[f(\bar{x}^{\tau})-f(x^{\star})]$:

[TABLE]

where we have used multiple times the convexity of $f$ (and of each $f_{j}$) and the subgradient boundedness (Assumption 1(C)). Let us now study the term $\sum_{i=1}^{N}\mathbb{E}[f_{i}(y_{i}^{\tau})-f_{i}(x^{\star})]$ in (31). By writing the optimality condition for the proximal mapping in the update (9), if $s_{i}^{\tau}=1$, one has

[TABLE]

Hence,

[TABLE]

Thus, from (33) and by using (27), one has that, if $s_{i}^{\tau}=1$, it holds

[TABLE]

while, if $s_{i}^{\tau}=0$,

[TABLE]

Now, by taking the expected value of $V_{i}^{\tau+1}$ conditioned to $\mathcal{S}^{\tau}$, one obtains

[TABLE]

and hence, by substituting (34) and (35) in (36),

[TABLE]

Notice now that

[TABLE]

Moreover, by noting that it holds that $\sum_{\ell=1}^{B}\mathbb{E}[\|g_{i,\ell}^{\tau}\|^{2}]=\mathbb{E}[\sum_{\ell=1}^{B}\|g_{i,\ell}^{\tau}\|^{2}]=\mathbb{E}[\|g_{i}^{\tau}\|^{2}]$ and $\sum_{\ell=1}^{B}U_{\ell}g_{i}^{\tau}=g_{i}^{\tau}$ and by substituting (38) in (37), we obtain

[TABLE]

where in the last inequality we used the separate convexity of $\nu_{\ell}$ from Assumptions 3 and the fact that $\mathbb{E}[\|g_{i}^{\tau}\|^{2}]\leq\bar{G}_{i}$ (Assumption 1(C)), and $\mathbb{E}[g_{i}^{\tau}]=\textsl{g}_{i}^{\tau}$ (Assumption 1(B)). Now, by summing over $i$,

[TABLE]

where in the last inequality we used the convexity of $f_{i}$. Taking the expected value conditioned to $\mathcal{S}^{0}$ on both sides of (40), using $\mathbb{E}[V_{i}^{\tau+1}{\mid}\mathcal{S}^{\tau}]{=}\mathbb{E}[V_{i}^{\tau+1}{\mid}\mathcal{S}^{\tau},\mathcal{S}^{0}]$ and Lemma 4, gives

[TABLE]

and, rearranging the terms,

[TABLE]

Now, by summing over $\tau$, and noting that $\mathbb{E}[V_{i}^{0}\mid\mathcal{S}^{0}]=V_{i}^{0}$,

[TABLE]

Moreover, by taking the expected value over $\mathcal{S}^{0}=\{\bm{x}^{0}\}$,

[TABLE]

Notice now that, since by definition $a_{m}^{\tau}\leq\alpha_{i}^{\tau}$ for all $i$ and $\tau$, we have

[TABLE]

Finally, by combining (30), (31), (41) and (42) one obtains (29). ∎

A similar result can be given also in terms of the sequences of local solution estimates $\{x_{i}^{t}\}$ as formalized in the next result.

Corollary 1**.**

Let Assumptions 1, 2, 3, 6 hold. Then,

[TABLE]

for all $i\in\{1,\ldots,N\}$.

Proof.

The proof follows the same line of the one of Theorem 1, by noting that

[TABLE]

so that in place of (30), one has

[TABLE]

for all $i\in\{1,\ldots,N\}$. ∎

The previous two results hold true without making any assumption on the local stepsize sequences. In the next two subsections the general result of Theorem 1 is specialized to the case of constant and diminishing stepsizes respectively.

5.2.1 Constant stepsizes

In the case of constant (local) stepsizes, convergence with a constant error is attained with an explicit sublinear convergence rate.

Theorem 2**.**

Let Assumptions 1, 2, 3, 4, 6 hold. Then, there exist constants $M\in(0,\infty)$ and $\mu_{M}\in(0,1)$ such that

[TABLE]

for all $i\in\{1,\ldots,N\}$, with

[TABLE]

Proof.

By exploiting Assumption 4, Lemma 10 and Lemma 8, from (43) one obtains

[TABLE]

which, by rearranging the terms, leads to

[TABLE]

thus concluding the proof. ∎

The previous result shows that, when constant stepsizes are employed, the value of $f_{\textsl{best}}(x_{i}^{t})$ converges to $f(x^{\star})$ plus a constant error, which can be retrieved from (44) by taking the limit for $t\to\infty$, i.e., $\lim_{t\to\infty}f_{\textsl{best}}(x_{i}^{t})-f(x^{\star})\leq S$ with the explicit expression for $S$ being

[TABLE]

It is worth noting that the bound decreases with the maximum stepsize $a_{M}$. Regarding the convergence rate, it is sublinear $O\left(Q/t\right)$. However, in the first iterations, the term $(\mu_{M})^{t}R$ can be dominant (if $|R|\gg Q$), thus leading to a linear rate at the beginning of the algorithm (as it will be shown in the numerical example). Notice that $Q$ (and hence the convergence rate) depends both on the number of blocks and on the local probabilities of being awake and drawing blocks. The local probabilities appear in $V^{0}$ and (implicitly) in the constant $\mu_{M}$. In fact, $\mu_{M}$ is related to the randomness of the matrices $W_{\ell}^{t}$, which depend on such probabilities. In particular notice that, if $B=1$ the rate is similar to those obtained in [27], in which the proximal mapping $\nu(a,b)=\frac{1}{2}\|a-b\|^{2}$ is used. Clearly, one may argue that the best rate is achieved by using a single block and hence, communicating in terms of blocks is useless. This is true only if we assume an infinite bandwidth to be available in the communication channels (i.e, transmitting the entire optimization variable or a single block of it requires the same amount of time). However, in typical real world scenarios this is not true and data that exceed the communication bandwidth are transmitted sequentially. If only one block fits the communication channel, our algorithm allows to perform an update at each communication round, while classical ones would need $B$ communication rounds per update. Moreover, in the proposed algorithm, typically, the local computation time at each iteration is not negligible, since a minimization problem is to be solved at every step (see (7)). Solving such an optimization problem on the entire optimization variable or on a single block of it clearly results in completely different computational times, which are clearly lower in the case of block-wise updates. Thus, the benefits of using block-wise updates and communications make the Distributed Block Proximal Method well suited for big-data optimization problems.

5.2.2 Diminishing stepsizes

In the case of diminishing (local) stepsizes, asymptotic convergence to the optimal cost can be reached.

Theorem 3**.**

Let Assumptions 1, 2, 3, 5, 6 hold. Then,

[TABLE]

for all $i\in\{1,\ldots,N\}.$

Proof.

The proof follows by taking the limit for $t\to\infty$, and using Assumption 5 and Lemma 12 in (43). ∎

Remark 2*.*

We point out that, similarly to, e.g., [37, 11], one can introduce a running averaging mechanism by defining $\hat{x}_{i}^{t}=\tfrac{1}{t}\sum_{\tau=1}^{t}x_{i}^{t}$ and provide the convergence results in terms of $f(\hat{x}_{i}^{t})-f(x^{\star})$, in place of $f_{\textsl{best}}(x_{i}^{t})-f(x^{\star})$. The convergence proof would follow almost the same line with some adjustments in (30)-(31) (see [11]). However, running averaging mechanisms typically lead to a significantly slower convergence rate. In light of this, we provided our results in terms of $f_{\textsl{best}}(x_{i}^{t})-f(x^{\star})$.

6 Numerical example

Consider a soft margin classification problem in which each agent $i\in\{1,\ldots,N\}$ has $m_{i}$ training samples $q_{i}^{1},\dots,q_{i}^{m_{i}}\in\mathbb{R}^{d}$ each of which has an associated binary label $b_{i}^{r}\in\{-1,1\}$ for all $r\in\{1,\ldots,m_{i}\}$. The goal of the network is to build a linear classifier from the training samples, i.e., to find a hyperplane of the form $\{z\in\mathbb{R}^{d}\mid\langle\theta,z\rangle+\theta_{0}=0\}$, with $\theta\in\mathbb{R}^{d}$ and $\theta_{0}\in\mathbb{R}$, which better separates the training data. Let us define $x=[\theta^{\top},\theta_{0}]^{\top}\in\mathbb{R}^{d+1}$ and $\hat{q}_{i}^{r}=[(q_{i}^{r})^{\top},1]^{\top}$. Then, the solution to this problem can be determined by solving the following SVM problem

[TABLE]

where $\lambda>0$ is the regularization weight. Problem (45) can be written in the form of problem (1) by defining $\xi_{i}^{r}=(\hat{q}_{i}^{r},b_{i}^{r})$ and

[TABLE]

for all $i\in\{1,\ldots,N\}$. Notice that, as long as each data $\xi_{i}^{r}$ is uniformly drawn from the dataset, Assumption 1(B) is satisfied.

In the next two sections we will test the Distributed Block Subgradient Method in the presented scenario, first on a synthetic dataset and then on real-world dataset composed of text documents. In both cases we consider a system with $N=48$ processors. The proposed distributed algorithm has been implemented by using the Python package DISROPT [51], and each processor has been assigned an agent.

6.1 Synthetic dataset

In order to show how the algorithm performs for different number of blocks, let us consider a relatively low-dimensional problem with $x\in\mathbb{R}^{50}$ and evaluate the algorithm performance for different number of blocks, namely $B\in\{1,2,5,10,50\}$. We generate a synthetic dataset (with $x\in\mathbb{R}^{50}$) composed of 240 points (taken from two clusters corresponding to labels $-1$ and $1$ respectively) and assign $5$ of them to each agent, i.e., $m_{1}=\dots=m_{N}=5$. Finally, we set $\lambda=0.1$, a common (constant) stepsize $\alpha=0.2$, $p_{i,\ell}=1/B$ for all $i$ and all $\ell$ and $s_{i}^{t}=1$ for all $i$ and all $t$. Regarding the communication graph, it has is generated according to an Erdős-Rènyi random model with connectivity parameter $p=0.3$. The corresponding weight matrix is built by using the Metropolis-Hastings rule. The evolution of the cost error adjusted with respect to the number of blocks is reported in Figure 1 for the considered block numbers. The results confirm the discussion carried out in Section 5.2.1 about the role of block communications. In fact, when normalizing the number of iterations with respect to the number of blocks, the convergence rates for the considered number of blocks are comparable. Moreover, the convergence rate exhibits the properties shown in Section 5.2.1. In fact, it is linear at the beginning and becomes sublinear after some iterations. Moreover, as expected when using constant stepsizes, convergence is reached with a constant error.

6.2 Text classification

Let us now consider a real-world scenario in which the local training samples are drawn from a dataset of texts. In particular, we pick the 20 newsgroups dataset, a dataset consisting of 18,846 newsgroup posts belonging to $20$ topics. Texts are represented by tf-idf on a dictionary of 130,107 words, so that each sample is a vector in $\mathbb{R}^{130,107}$. Agents have to learn to classify posts belonging to the class sci.med from the others. Thus, in order to perform a binary classification, we assign the label $1$ to samples belonging to the class sci.med, and $-1$ to all the other samples. In this scenario, the considered $48$ agents, are connected over a balanced directed graph, generated according to a binomial random model with connectivity parameter $p=0.5$ and each agent is awake with probability $p_{i,on}=0.95$. The entire dataset is split to assign almost the same number of samples to each agent. We run the algorithm for $4000$ iterations and for different number of blocks, namely $B\in\{10,10^{2},10^{3},10^{4}\}$. Moreover, we set $\lambda=0.001$ in problem (45), and we select a common (constant) stepsize $\alpha=0.5$ and $p_{i,\ell}=1/B$ for all $i$ and all $\ell$. Differently from the previous example over synthetic data, in this case computing the exact (centralized) solution of the considered problem is computationally intractable, due to the high dimension of the decision variable ($n=$130,107) and the large number of samples ($\sum_{i=1}^{N}m_{i}=$18,146). Thus, the performance of the algorithm are evaluated in terms of the accuracy $\Psi$ of the average of the produced solution estimates $\bar{x}^{t}$, i.e., the number of samples of the dataset that are correctly classified through the hyperplane defined by $\bar{x}^{t}$. The results are reported in Figure 2.

7 Conclusions

In this paper, we introduced a class of distributed block proximal algorithms for solving stochastic big-data convex optimization problems over networks. In the addressed optimization set-up the dimension of the decision variable is very high and the (stochastic) cost function may be nonsmooth. The main strength of the proposed algorithms is that agents in the network can communicate a single block of the optimization variable per iteration. Under the assumption of diminishing stepsizes, we showed that the agents in the network asymptotically agree on a common solution which is cost-optimal in expected value. When employing constant stepsizes approximate convergence is attained with a constant error on the optimal cost and an explicit convergence rate is provided. Special instances of the algorithm are presented for particular classes of problems. Finally, the proposed algorithm, has been numerically evaluated on a distributed classification problem over both a synthetic dataset and a real, high-dimensional, text document dataset.