Penalized Push-Sum Algorithm for Constrained Distributed Optimization with Application to Energy Management in Smart Grid

TL;DR

This paper introduces a penalty-based push-sum algorithm for distributed convex constrained optimization in multi-agent networks, demonstrating its effectiveness in energy management within smart grids.

Contribution

It adapts the push-sum protocol to handle convex constraints using penalty functions, with proven convergence and practical application to smart grid energy management.

Findings

Proven convergence of the algorithm under certain assumptions.

Effective application to distributed energy management in smart grids.

Advantages over existing distributed optimization methods.

Abstract

We study distributed convex constrained optimization on a time-varying multi-agent network. Each agent has access to its own local cost function, its local constraints, and its instant number of out-neighbors. The collective goal is to minimize the sum of the cost functions over the set of all constraints. We utilize the push-sum protocol to be able to solve this distributed optimization problem. We adapt the push-sum optimization algorithm, which has been studied in context of unconstrained optimization so far, to convex constrained optimization by introducing an appropriate choice of penalty functions and penalty parameters. Under some additional technical assumptions on the gradients we prove convergence of the distributed penalty-based push-sum algorithm to the optimal value of the global objective function. We apply the proposed penalty-based push-sum algorithm to the problem of distributed energy management in smart grid and discuss the advantages of this novel procedure in comparison with existing ones.

Tables

Table 1. TABLE I: Optimal 𝒑 ∗ superscript 𝒑 \bm{p}^{*} vs algorithm output after 3 × 10 4 3 superscript 10 4 3\times 10^{4} iterations.

	${𝒑}^{\ast }$	$𝒑$
$p_1^G$	81.98	81.69
$p_2^G$	124.80	122.75
$p_1^D$	100.34	99.61
$p_2^D$	100.00	99.64

Equations

$$\min F(\mathbf{z})$$

$$\text{mbox}s.t.c_1(\mathbf{z})$$

$$G(t:t+B)=([n],E(t)\cup E(t+1)\cup \cdots \cup E(t+B-1)),$$

$${\Psi }_i(\mathbf{z})$$

$$g(u)$$

$$\underset{\mathbf{z}\in {\mathbb{R}}^d}{\min }{F}_t(\mathbf{z})=F(\mathbf{z})+r_t\Psi (\mathbf{z})=\underset{\mathbf{z}\in {\mathbb{R}}^d}{\min }\sum _{i=1}^n[F_i(\mathbf{z})+r_t{\Psi }_i(\mathbf{z})],$$

$${\mathbf{w}}_i(t+1)$$

$$y_i(t+1)$$

$${\mathbf{z}}_i(t+1)$$

$${\mathbf{x}}_i(t+1)$$

$$\displaystyle a_{t}\leq a_{s}\mbox{ for all $t\geq s$, }\,\sum_{t=0}^{\infty}a_{t}=\infty,$$

$$r_t\ge 1,r_t\to \infty ,$$

$$\sum _{t=0}^{\infty }{a}_t^2{r}_t^3<\infty ,r_{t+1}-r_t=o(a_t)\text{mbox}as{a}_t\to 0.$$

$$\bar{\mathbf{x}}(t+1)=\bar{\mathbf{x}}(t)-$$

$$=\bar{\mathbf{x}}(t)-$$

$$-$$

$$\mathbf{q}(t,\bar{\mathbf{x}}(t))$$

$$\mathbf{p}(t,\bar{\mathbf{x}}(t))$$

$$\Vert \mathbf{q}(t,\bar{\mathbf{x}}(t))\Vert$$

$$\Vert \mathbf{p}(t,\bar{\mathbf{x}}(t))\Vert$$

$$\sum _{t=0}^{\infty }{a}_t{r}_t^{3}q(t)<\infty ,\sum _{t=0}^{\infty }{a}_t{r}_t^{3}p(t)<\infty .$$

$$\Vert \mathbf{q}(t,\bar{\mathbf{x}}(t))\Vert$$

$$\le \frac{l}{n}\sum _{i=1}^n\Vert {\mathbf{z}}_i(t+1)-\bar{\mathbf{x}}(t)\Vert .$$

$$\sum _{t=1}^{\infty }{b}_t{a}_t\Vert {\mathbf{f}}_i({\mathbf{z}}_i(t+1))+r_t{\mathbf{\psi }}_i({\mathbf{z}}_i(t+1)){\Vert }_1.$$

$$\sum _{t=1}^{\infty }{b}_t{a}_t\Vert {\mathbf{f}}_i({\mathbf{z}}_i(t+1))+r_t{\mathbf{\psi }}_i({\mathbf{z}}_i(t+1)){\Vert }_1<\infty ,$$

$$\sum _{t=0}^{\infty }{a}_t{r}_t^{3}q(t)=l\sum _{t=0}^{\infty }\frac{a_t{r}_t^3}{n}\sum _{i=1}^n\Vert {\mathbf{z}}_i(t+1)-\bar{\mathbf{x}}(t)\Vert$$

$$\tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t))=\mathbf{f}(\bar{\mathbf{x}}(t))+r_t\mathbf{\psi }(\bar{\mathbf{x}}(t))+\mathbf{q}(t,\bar{\mathbf{x}}(t))+r_t\mathbf{p}(t,\bar{\mathbf{x}}(t)),$$

$$\Psi$$

$$+a_t[⟨\mathbf{\psi }(\bar{\mathbf{x}}(t)),\tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t))⟩-⟨\mathbf{\psi }(\tilde{\mathbf{x}}(t)),\tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t))⟩].$$

$$⟨\mathbf{\psi }(\bar{\mathbf{x}}(t)),$$

$$\le \Vert \tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t))\Vert \Vert \mathbf{\psi }(\bar{\mathbf{x}}(t))-\mathbf{\psi }(\tilde{\mathbf{x}}(t))\Vert$$

$$\le \Vert \tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t))\Vert mn\Vert \bar{\mathbf{x}}(t)-\tilde{\mathbf{x}}(t)\Vert$$

$$\le \theta mn{a}_t\Vert \tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t)){\Vert }^2,$$

$$\Vert \tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t)){\Vert }^2\le \Vert \mathbf{f}(\bar{\mathbf{x}}(t)){\Vert }^2$$

$$+r_t^2\Vert \mathbf{\psi }(\bar{\mathbf{x}}(t)){\Vert }^2+\Vert \mathbf{q}(t,\bar{\mathbf{x}}(t)){\Vert }^2+r_t^2\Vert \mathbf{p}(t,\bar{\mathbf{x}}(t)){\Vert }^2$$

$$+2\Vert \mathbf{f}(\bar{\mathbf{x}}(t))\Vert r_t\Vert \mathbf{\psi }(\bar{\mathbf{x}}(t))\Vert +2\Vert \mathbf{f}(\bar{\mathbf{x}}(t))\Vert \Vert \mathbf{q}(t,\bar{\mathbf{x}}(t))\Vert$$

$$+2r_t\Vert \mathbf{q}(t,\bar{\mathbf{x}}(t))\Vert \Vert \mathbf{p}(t,\bar{\mathbf{x}}(t))\Vert +2r_t\Vert \mathbf{\psi }(\bar{\mathbf{x}}(t))\Vert \Vert \mathbf{q}(t,\bar{\mathbf{x}}(t))\Vert$$

$$+2r_t^2\Vert \mathbf{\psi }(\bar{\mathbf{x}}(t))\Vert \Vert \mathbf{p}(t,\bar{\mathbf{x}}(t))\Vert +2\Vert \mathbf{f}(\bar{\mathbf{x}}(t))\Vert \Vert \mathbf{p}(t,\bar{\mathbf{x}}(t))\Vert$$

$$\le k_1(\frac{1}{2}+q(t)+r_{t}p(t))+k_2(\frac{1}{2}{r}_t^2+r_{t}q(t)+r_t^{2}p(t))$$

$$+k_3{r}_t+q^2(t)+r_t^2{p}^2(t)+2r_{t}p(t)q(t)=g_0(t)$$

Full text

I Introduction

Due to emergence of large-scaled networked systems with limited information, distributed multi-agent optimization problems have gained a lot of attention recently. In such systems a number of agents, represented by nodes over some communication graph, aim to optimize a global objective by taking only the local information into account. Beside the various applications of distributed optimization such as robust sensor network control [8], signal processing [14], network routing [5], and machine learning [16, 15], an important and promising area of applicability is energy management of future smart grid [3, 9, 17]. Smart grid is equipped with advanced communication technologies enabling efficient and distributed energy management between the grid’s users [17, 7]. However, from technical point of view, it is important to keep communication costs limited and choose a communication protocol that would require minimal coordination between the agents and stays robust against changes in the network topology [11]. That is why, in this paper, we develop a communication-based optimization algorithm with the desired features mentioned above.

For this purpose we utilize the push-sum communication protocol. This protocol was initially introduced in [2] and used in [15] for distributed optimization. The push-sum protocol is applicable to time-dependent network topology and it can overcome the restrictive assumptions on the communication graph structure such as double stochastic communication matrices [2, 15]. The work [4] studied this algorithm over directed and time-varying communication in the case of well-behaved convex functions. The authors in [13] extended the results to a broader class of non-convex functions. However, all works on the push-sum algorithm presented in the literature so far dealt with unconstrained optimization. As in many applications, including energy management in smart grid, agents face a number of constraints. In this paper, we adapt the push-sum algorithm to the case of convex constrained optimization by introducing an appropriate choice of penalty functions and penalty parameters. Under some standard technical assumptions, we prove convergence of the resulting procedure to the optimal value of the system’s objective function.

Another contribution of this paper consists in the application of the proposed procedure to the problem of energy management in smart grid. In contrast to the recent communication-based procedure proposed in [17], the penalty-based push-sum algorithm presented in this work is based on a time-dependent directed communication topology with column-stochastic matrices, where each agent merely needs to know the current number of its out-neighbors to define the elements of the communication matrix at each iteration. Moreover, it uses only one communication step per iteration, whereas the procedure in [17] requires two communication steps per iteration, with a row-stochastic and a column-stochastic communication matrix at the consequent communication iterations. Thus, the proposed penalty-based push-sum algorithm keeps communication costs cheaper and is able to adapt to the changes in the communication topology.

The paper is organized as follows. In Section II we introduce the penalty-based push-sum algorithm and prove its convergence. Section III deals with formulation of the general non-convex energy management problem in smart grid, presents its convex reformulation for which the penalty-based push-sum procedure can be applied, and demonstrates some simulation results. Section IV concludes the paper.

Notations. We will use the following notations throughout this paper: We denote the set of integers by $\mathbb{Z}$ and the set of non-negative integers by $\mathbb{Z}^{+}$. For the metric $\rho$ of a metric space $(X,\rho(\cdot))$ and two subsets $B_{1}\subset X$ and $B_{2}\subset X$, we let $\rho(B_{1},B_{2})=\max\{\sup_{x\in B_{1}}\inf_{y\in B_{2}}\rho(x,y),\sup_{y\in B_{2}}\inf_{x\in B_{1}}\rho(x,y)\}$. We denote the set $\{1,\ldots,n\}$ by $[n]$. We use boldface to distinguish between the vectors in a multi-dimensional space and scalars. We denote the dot product of two vectors $\mathbf{a}$ and $\mathbf{b}$ by $\langle\mathbf{a},\mathbf{b}\rangle$. $\|\cdot\|$ denotes the standard Euclidean norm, whereas $\|\cdot\|_{l_{1}}$ is used to denote $l^{1}$-norm in the vector space. Throughout this work, all time indices such as $t$ belong to $\mathbb{Z}^{+}$. For vectors $\mathbf{v}_{i}\in X^{d}$, $i\in[n]$, of elements in some vector space $X$ (over $\mathbb{R}$), we let $\bar{\mathbf{v}}=\frac{1}{n}\sum_{i=1}^{n}\mathbf{v}_{i}$. We say the function $F:\mathbb{R}^{d}\to\mathbb{R}$ be inf-compact, if the set $\{\mathbf{x}\in\mathbb{R}^{d}:F(\mathbf{x})\leq A\}$ is compact for all $A\in\mathbb{R}$. The function $1_{\{A\}}(x)$ denotes the indicator of the set $A$ ($1_{\{A\}}(x)=1,$ if $x\in A$ and $1_{\{A\}}(x)=0,$ otherwise). The notation $o(x)$ as $x\to x_{0}$ is for some function $f(x)$ such that $\lim_{x\to x_{0}}\frac{f(x)}{x}=0$.

II Push-sum Algorithm for Distributed Constrained Optimization

In this section we adapt the push-sum algorithm to the case of constrained convex optimization and prove convergence of the resulting procedure.

II-A Problem Formulation and Adapted Push Sum Algorithm

Let us consider the following general problem:

[TABLE]

where $F_{i}:\mathbb{R}^{d}\to\mathbb{R}$, $c_{i}:\mathbb{R}^{d}\to\mathbb{R}$, $i=1,\dots,n$, are some differentiable convex functions. Let $\mathbf{f}_{i}$ denote the gradient of the function $F_{i}$, $i\in[n]$, $\mathbf{f}=\sum_{i=1}^{n}\mathbf{f}_{i}$. This problem is formulated in a multi-agent system consisting of $n$ agents. Each agent $i$ has access to its local cost function $F_{i}$ and its local constraint described by the inequality $c_{i}(\mathbf{z})\leq 0$111For the sake of notation simplicity, we assume that the local constraint of each agent $i$ is expressed by only one function $c_{i}$. The analysis below is applicable to problems, where agents have more than one constraint function.. By the set $S$ we denote the set of solutions for (1). By $F^{*}$ we denote the optimal value of the objective function $F$ in the problem (1).

At each time $t$, node $i$ can only communicate to its out-neighbors in some directed graph $G(t)$, where the graph $G(t)$ has the vertex set $[n]$ and the edge set $E(t)$. We use $N^{in}_{i}(t)$ and $N^{out}_{i}(t)$ to denote the in- and out-neighborhoods of node $i$ at time $t$. Each node $i$ is always considered to be an in- and out-neighbor of itself. We use $d_{i}(t)$ to denote the out-degree of node $i$, and we assume that every node $i$ knows its out-degree at every time $t$. The goal of the agents is to solve distributively the constrained minimization problem (1). We introduce the following standard definition for the sequence $G(t)$.

Definition 1**.**

We say that a sequence of graphs $\{G(t)\}$ is $B$-strongly connected, if, for any time $t\geq 0$, the graph

[TABLE]

is strongly connected. In other words, the union of the graphs over every $B$ time intervals is strongly connected.

In the following analysis we assume that the sequence of the communication graphs $\{G(t)\}$ under consideration is $B$-strongly connected, which guarantees enough information “mixing” during communication between agents over time.

To deal with the problem described above, we aim to develop a distributed optimization procedure based on the push-sum protocol [2]. However, this protocol uses specific ratios of local agents’ values to cancel out the effect of information imbalances caused by limited agents’ knowledge of their neighborhoods in time-dependent and directed communication network[2, 4]. That is why optimization methods based on projection onto the set of constraints cannot be applied here, as they violate the balance properties guaranteed by taking the corresponding ratio. To overcome this limitation and to incorporate the constraints of the problem (1) into the optimization algorithm, we leverage the idea of penalty function methods [6]. We choose the convex penalty functions $\{\Psi_{i}(\mathbf{z})\}_{i}$ such that222Other candidates for penalty function can be found in [1], Chapter 5.

[TABLE]

Let $\Psi=\sum_{i=1}^{n}\Psi_{i}$, $\bm{\psi}_{i}(\mathbf{z})$ denote the gradient of the function $\Psi_{i}(\mathbf{z})$, $\bm{\psi}=\sum_{i=1}^{n}\bm{\psi}_{i}$. Note that for each $i\in[n]$ the vector-function $\bm{\psi}_{i}(\mathbf{z})$ is uniformly bounded over $\mathbb{R}^{d}$, given that $\nabla c_{i}(\mathbf{z})$ is uniformly bounded over $\mathbb{R}^{d}$. By adding the penalty function $\Psi$ to the objective function $F$ in (1), we obtain the following unconstrained penalized optimization problem:

[TABLE]

where $r_{t}$ is some positive penalty parameter. Note that as the functions $F_{i}$ and $\Psi_{i}$ are convex for all $i$ and $r_{t}>0$, the unconstrained problem above is convex. Let $S_{t}$ denote the set of solutions for (5). The connection between the penalized unconstrained problem (5) and the initial constrained one in (1) is shown in the following proposition (see [6]):

Proposition 1**.**

Let the function $F$ be inf-compact and $r_{t}\to\infty$ as $t\to\infty$. Then $S_{t}$ and $S$ are not empty and $S_{t}$ converges to $S$ as $t$ goes to infinity, namely $\lim_{t\to\infty}\rho(S,S_{t})=0$. Moreover, $\lim_{t\to\infty}F_{t}^{*}=F^{*}$, where $F_{t}^{*}=\min_{\mathbf{z}\in\mathbb{R}^{d}}F_{t}(\mathbf{z})$.

Next, we apply the push-sum algorithm from [4] to the penalized problem (5). We proceed with the formal algorithm formulation. At every moment of time $t\in\mathbb{Z}^{+}$ each node $i$ maintains vector variables $\mathbf{z}_{i}(t)$, $\mathbf{x}_{i}(t)$, $\bm{w}_{i}(t)\in\mathbb{R}^{d}$, as well as a scalar variable $y_{i}(t)$ such that $y_{i}(0)=1$ for all $i\in[n]$. These quantities are updated as follows:

[TABLE]

where $a_{t}\geq 0$ is a time-dependent step size for all $t$. The version of the push-sum algorithm above corresponds to the one proposed in [4], where the optimization step (6d) is augmented by the penalty term $r_{t}\bm{\psi}_{i}(\mathbf{z}_{i}(t+1))$. Note that the algorithm above is based on a time-dependent communication topology, where each agent $i$ merely needs to know its current out-degree $d_{i}(t)$ to follow the algorithm’s steps.

II-B Convergence of the Algorithm

In what follows, we analyze the convergence property of the algorithm (6) under the following assumptions regarding the gradient functions.

Assumption 1**.**

The gradients $\mathbf{f}_{i}$ and $\nabla c_{i}$ are uniformly bounded over $\mathbb{R}^{d}$ for all $i\in[n]$.

Remark 1**.**

Since $\mathbf{f}_{i}$ is assumed to be bounded for any $i\in[n]$, there exists a positive constant $L_{i}$ such that $\|\mathbf{f}_{i}(\mathbf{z})\|\leq L_{i}$ for any $\mathbf{z}\in\mathbb{R}^{d}$. Let $L=\max_{i\in[n]}L_{i}$. Moreover, due to the bounded gradients $\nabla c_{i}$, the gradient function $\bm{\psi}_{i}$ is bounded for any $i\in[n]$ (see (3)). Thus, there exists a positive constant $M_{i}$ such that $\|\bm{\psi}_{i}(\mathbf{z})\|\leq M_{i}$ for any $\mathbf{z}\in\mathbb{R}^{d}$. Let $M=\max_{i\in[n]}M_{i}$.

Assumption 2**.**

The gradients $\mathbf{f}_{i}$ and $\nabla c_{i}$ are Lipschitz continuous over $\mathbb{R}^{d}$ for all $i\in[n]$.

Remark 2**.**

According to the choice of the penalty functions in (3), Lipschitz continuity of $\nabla c_{i}$ over $\mathbb{R}^{d}$ implies Lipschitz continuity of $\bm{\psi}_{i}$ over $\mathbb{R}^{d}$ for all $i\in[n]$. Thus, given Assumption 2, there exist positive constants $l_{i}$ and $m_{i}$ such that $\|\mathbf{f}_{i}(\mathbf{z}_{1})-\mathbf{f}_{i}(\mathbf{z}_{2})\|\leq l_{i}\|\mathbf{z}_{1}-\mathbf{z}_{2}\|$ and $\|\bm{\psi}_{i}(\mathbf{z}_{1})-\bm{\psi}_{i}(\mathbf{z}_{2})\|\leq l_{i}\|\mathbf{z}_{1}-\mathbf{z}_{2}\|$ for any $\mathbf{z}_{1},\mathbf{z}_{2}\in\mathbb{R}^{d}$ and all $i\in[n]$ respectively. Let $l=\max_{i\in[n]}l_{i}$ and $m=\max_{i\in[n]}m_{i}$.

Moreover, we make the following assumption regarding the parameters $a_{t}$, $r_{t}$.

Assumption 3**.**

[TABLE]

Remark 3**.**

Note that the conditions (7a), (7c) imply that $a_{t}\to 0$ as $t\to\infty$. Appropriate sequences $\{a_{t}\}$ and $\{r_{t}\}$ that meet the assumption above can be, for example, $a_{t}=\frac{1}{t^{0.5+b}}$, $r_{t}=t^{0.25b}$, where $0<b<0.4$.

According to the procedure (6), the running average of $\{\mathbf{x}_{i}(t)\}_{i\in[n]}$, namely $\bar{\mathbf{x}}(t)=\frac{1}{n}\sum_{i=1}^{n}\mathbf{x}_{i}(t)$, fulfills the following iterations (see also [4]):

[TABLE]

Some helpful results that will be used in the convergence analysis are presented in Appendix (see Theorems 2 and 3). In particular, Theorem 2(a) implies that all $\mathbf{x}_{i}(t+1)$, $i\in[n]$, converge with time to their running average $\bar{\mathbf{x}}(t)$, given that Assumptions 1 and 3 hold.

Further we utilize the following notations:

[TABLE]

We will use the following lemma which bounds the norms of the vectors $\mathbf{q}(t,\bar{\mathbf{x}}(t))$ and $\mathbf{p}(t,\bar{\mathbf{x}}(t))$ introduced above.

Lemma 1**.**

Let Assumptions 1-3 hold. Then, there exists $q(t)$ and $p(t)$ such that the following holds for the process (6):

[TABLE]

[TABLE]

such that

[TABLE]

Proof.

Due to Assumption 2,

[TABLE]

Let $q(t)=\frac{l}{n}\sum_{i=1}^{n}\|\mathbf{z}_{i}(t+1)-\bar{\mathbf{x}}(t)\|$. Next, let us consider the series

[TABLE]

If $b_{t}=a_{t}r_{t}^{3}$,

[TABLE]

as $\mathbf{f}_{i}(\mathbf{z}_{i}(t+1))$ and $\bm{\psi}_{i}(\mathbf{z}_{i}(t+1))$ are bounded, (7b) and (7c) hold for $a_{t}$ and $r_{t}$. Thus, we can use Theorem 2 from Appendix to conclude that

[TABLE]

Analogously, one can show that $\sum_{t=0}^{\infty}a_{t}r^{3}_{t}p(t)<\infty$. ∎

Remark 4**.**

Note that due to the choice of the parameters in Assumption 3 and the fact that under Assumptions 1-3 both $p(t)$ and $q(t)$ tend to 0 as $t\to\infty$ (see Theorem 2 in Appendix), $\sum_{t=0}^{\infty}a_{t}^{q_{1}}r^{q_{2}}_{t}q^{2}(t)<\infty$, $\sum_{t=0}^{\infty}a_{t}^{p_{1}}r^{p_{2}}_{t}p^{2}(t)<\infty$, and $\sum_{t=0}^{\infty}a_{t}^{2}r^{2}_{t}p(t)q(t)<\infty$ for all integers $q_{1},p_{1}\geq 1$, $q_{2},p_{2}\in\{0,1,2,3\}$.

Now we state the main result for the penalty-based push-sum algorithm (6).

Theorem 1**.**

Let the function $F$ in the problem (1) be inf-compact. Let Assumptions 1-3 hold. Then all local variables $\mathbf{z}_{i}(t+1)$, $i\in[n]$, in the procedure (6) reach a consensus as $t\to\infty$ and each limit point of this consensus corresponds to a solution to the problem (1), given that the sequence of the communication graphs $\{G(t)\}$ under consideration is $B$-strongly connected.

Proof.

First, we will show that $\lim_{t\to\infty}\Psi(\bar{\mathbf{x}}(t))=0$. In particular, it will mean that all limit points of $\{\bar{\mathbf{x}}(t)\}$ belong to the feasible set $C=\{\mathbf{z}\in\mathbb{R}^{d}\,|\,c_{i}(\mathbf{z})\leq 0,i\in[n]\}$. Taking the Mean-value Theorem and relation (8) into account, and using the notation

[TABLE]

we get that for some $\theta\in[0,1]$ and $\tilde{\mathbf{x}}(t)=\bar{\mathbf{x}}(t)-\theta a_{t}\tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t))$

[TABLE]

According to Remark 2 and the Cauchy-Schwarz inequality, we obtain

[TABLE]

where $m$ is the constant defined in Remark 2.

Next, using Remark 2, Lemma 1, and due to the Cauchy-Schwarz inequality, we get

[TABLE]

for some positive constants $k_{1},k_{2},$ and $k_{3}$. Finally, using the definition of $\tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t))$ in (12) and the Cauchy-Schwarz inequality again, we obtain for some positive constants $k_{4}$ and $k_{5}$ that

[TABLE]

where the first inequality is due to the bounded $\|\bm{\psi}\|$ and the last inequality is due to the fact that $0<r_{t}\to\infty$ as $t\to\infty$ and $\|\mathbf{f}\|$ are bounded.

By substituting (15)-(26) to (13) we can write:

[TABLE]

where $g(t)=a^{2}_{t}(g_{0}(t)+k_{4}q(t)+k_{5}r_{t}p(t))$. According to the choice of $a_{t}$ and $r_{t}$, Lemma 1, and Remark 4, we obtain that $\sum_{t=0}^{\infty}g(t)<\infty$. Thus, using the well-known result on the sequences of non-negative variables presented in Theorem 3 (see Appendix), we conclude that $\lim_{t\to\infty}\Psi(\bar{\mathbf{x}}(t))$ exists, is finite, and $\sum_{t=0}^{\infty}a_{t}\|\bm{\psi}(\bar{\mathbf{x}}(t))\|^{2}<\infty$. Thus, $\liminf_{t\to\infty}\|\bm{\psi}(\bar{\mathbf{x}}(t)\|=0$, since $\sum_{t=0}^{\infty}a_{t}=\infty$ (see (7a)). It implies existence of a subsequence $\{t_{k}\}\subseteq\{t\}$ such that $\lim_{k\to\infty}\|\bm{\psi}(\bar{\mathbf{x}}(t_{k})\|=0$ and, as $\|\bm{\psi}(\mathbf{z})\|=0$ if and only if $\Psi(\mathbf{z})=0$, we conclude that $\lim_{k\to\infty}\Psi(\bar{\mathbf{x}}(t_{k}))=0$. Thus,

[TABLE]

Next, let us notice that $F(\bar{\mathbf{x}}(t+1))=F_{t}(\bar{\mathbf{x}}(t+1))-r_{t}\Psi(\bar{\mathbf{x}}(t+1))=F_{t+1}(\bar{\mathbf{x}}(t+1))-r_{t+1}\Psi(\bar{\mathbf{x}}(t+1))$. Hence, taking into account that $\Psi(\bar{\mathbf{x}}(t))=o(1)$ as $t\to\infty$ (see (31)), $r_{t+1}-r_{t}=o(a_{t})$, $a_{t}\to 0$ as $t\to\infty$, and using Mean-value Theorem, we get

[TABLE]

where ${\mathbf{x}}^{\prime}(t)=\bar{\mathbf{x}}(t)-\beta a_{t}\tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t))$ for some $\beta\in[0,1]$.

According to Assumption 2, there exists some $l_{1}>0$ such that

[TABLE]

where for the first inequality we used the Cauchy-Schwarz inequality and the last inequality is due to (19). Hence, due to the Cauchy-Schwarz inequality and the definition of $\tilde{\mathbf{f}}(t,\bar{\mathbf{x}}(t))$ in (12), we obtain from (32) that

[TABLE]

where

[TABLE]

Thus, $\sum_{t=0}^{\infty}g_{1}(t)<\infty$, due to Assumption 1, the choice of $a_{t}$ and $r_{t}$, and Lemma 1 (see Remark 4). Hence, according to Theorem 3 from Appendix, we can conclude that

[TABLE]

Moreover, $\sum_{t=1}^{\infty}a_{t}\|\nabla F_{t}(\bar{\mathbf{x}}(t))\|^{2}<\infty$, which, due to (7a), implies $\liminf_{t\to\infty}\|\nabla F_{t}(\bar{\mathbf{x}}(t))\|=0$. Let us choose a subsequence $\{t_{k}\}\subseteq\{t\}$ such that $\lim_{k\to\infty}\|\nabla F_{t_{k}}(\bar{\mathbf{x}}(t_{k}))\|=0$. Due to convexity of $F_{t}$ over $\mathbb{R}^{d}$ for all $t$, the last limit implies that $\lim_{k\to\infty}[F_{t_{k}}(\bar{\mathbf{x}}(t_{k}))-F^{*}_{t_{k}}]=0$, where $F^{*}_{t_{k}}=\min_{\bm{x}\in\mathbb{R}^{d}}F_{t_{k}}(\bm{x})$. Next, due to Proposition 1 and as $r_{t}\to\infty$, we conclude that $\lim_{k\to\infty}F_{t_{k}}(\bar{\mathbf{x}}(t_{k}))-F^{*}=0$, which together with (36) implies that $\lim_{t\to\infty}F_{t}(\bar{\mathbf{x}}(t))=F^{*}$ and, hence, every limit point of $\bar{\mathbf{x}}(t)$ is a solution to the problem (1). Finally, by invoking Theorem 2(a), we conclude the result. ∎

Remark 5**.**

Note that if, additionally to the conditions in Theorem 1, the function $F$ is assumed to be strictly convex, then there exists a unique solution $\mathbf{z}^{*}$ to the problem (1). In this case, Theorem 1 implies convergence of all $\mathbf{z}_{i}(t)$ evolving according to the algorithm (6) to this optimum $\mathbf{z}^{*}$.

III Applications in Energy Management

We consider a problem of energy management formulated and analyzed in [17]. Let $\mathcal{N}_{g}$ and $\mathcal{N}_{d}$ be the sets of distributed generators and responsive demands in a power grid with $N_{g}$ and $N_{d}$ elements respectively. Let $N=N_{g}+N_{d}$. A directed connected time-dependent graph $G(t)=([N],\mathcal{E}(t))$ is used to represent the communication topology of the network in the grid, where $[N]=\mathcal{N}_{g}\cup\mathcal{N}_{d}$ is the set of the nodes containing generators and demands and $\mathcal{E}(t)\subseteq[N]\times[N]$ is the edge set. Note that $(j,i)\in\mathcal{E}(t)$ if and only if the node $i\in[N]$ can receive information from node $j\in[N]$ at time $t$. In contrast to the previous works [10, 17], in this paper we focus on a broader class of the communication topology containing time-dependent graphs and requiring each user to know only its current out-degree to construct an appropriate communication matrix. For this purpose, we will apply the penalty-based push-sum algorithm introduced and analyzed in Section II to the distributed optimization formulated below.

We consider the following generation and demand capacities in the system:

[TABLE]

The cost function $C_{i}:\mathbb{R}\to\mathbb{R}$ of each generator $i\in\mathcal{N}_{g}$ is:

[TABLE]

where $a_{i},b_{i},c_{i}$ are positive fitting parameters. Thus, the cost functions are strongly convex functions.

The utility function $U_{j}:\mathbb{R}\to\mathbb{R}$ of each demand $j\in\mathcal{N}_{d}$ has the following properties:

[TABLE]

Thus, the utility functions are concave functions.

Let $\bm{p}\in\mathbb{R}^{N}$ be the vector with coordinates $p_{i}$, $i\in[N]$. The goal in the power grid is to solve distributively the following energy management problem333For more details on the problem formulation see [17].:

[TABLE]

[TABLE]

In the problem above the constraint (41a) corresponds to the balance between the generated and the demanded power in the network, where each parameter $l_{i}$, $i\in\mathcal{N}_{g}$, corresponds to the coefficient of the transmission losses induced by the generator $i$ and satisfies $0\leq l_{i}<a_{i}$. Note that the problem (41) is non-convex due to the non-convex constraint defined by (41a).

III-A Problem reformulation with constraints based on local information

To implement the distributed penalized push-sum algorithm to the energy management problem (41), we need to find its appropriate convex reformulation such that any solution to this reformulation provides a solution for (41). Moreover, as the constraint (41a) contains the information on the “loss” parameter $l_{i}$ of each $i\in\mathcal{N}_{g}$, we aim to find a reformulation, where no constraint requires knowledge about the local properties of other nodes in the network. First of all, let us notice that the problem (41) is equivalent to the following one:

[TABLE]

[TABLE]

where $\bm{v}\in\mathbb{R}^{N_{g}}$ is the vector with coordinates $v_{i}$, $i\in\mathcal{N}_{g}$. Thus, the strategy $p_{i}$ of each generator is augmented by the auxiliary parameter $v_{i}$. However, the problem (42) is still non-convex due to the constraints in (42d). Following the idea in [17], we present a new reformulation, where each non-convex equality constraint is replaced by the corresponding convex inequality one. Thus, we obtain the convex optimization problem

[TABLE]

[TABLE]

Next, we establish the relation between the convex problem (43) and the initial one (41). This will be done under the following two technical assumptions.

Assumption 4**.**

The upper and low bounds for the feasible power generation and demand (see (41b) and (41c)) satisfy

[TABLE]

The assumption above repeats the sufficient condition for an appropriate convex reformulation of the problem (41) presented in [17] (see also Remark 1 in [17]). The next assumption is the Slater constraint qualification for the reformulated problem (43). It will enable the relation analysis based on the Karush-Kuhn-Tacker conditions for the optimal primal dual pair of the problem (43).

Assumption 5**.**

There exists a feasible point $(\hat{\bm{p}},\hat{\bm{v}})$ for the problem (43) such that $\sum_{i\in\mathcal{N}_{g}}(\hat{p}_{i}-\hat{v}_{i})=\sum_{j\in\mathcal{N}_{d}}\hat{p}_{j}$, $p_{i}^{m}<\hat{p}_{i}<P_{i}^{M}$, $i\in\mathcal{N}_{g}$, $p_{j}^{m}<\hat{p}_{j}<P_{j}^{M}$, $j\in\mathcal{N}_{d}$, $\hat{v}_{i}>l_{i}\hat{p}_{i}^{2}$, $i\in\mathcal{N}_{g}$.

The next proposition states the desired relation between the initial problem (41) and the convex one (43) above.

Proposition 2**.**

Let Assumptions 4 and 5 hold. Then any solution to the problem (43) is a solution to the problem (41).

Proof.

See Appendix. ∎

The optimization problem (43) can be considered a particular case of the general distributed optimization problem (1). Indeed, let $\mathbf{z}=(\bm{p},\bm{v})$ be the vector of joint strategies of the generators and responsive demands in the network, $F_{i}(\mathbf{z})=C_{i}(p_{i})$, if $i\in\mathcal{N}_{g}$, $F_{j}(\mathbf{z})=-U_{i}(p_{j})$, if $j\in\mathcal{N}_{g}$, and the constraints (43a)-(43d) be distributed over these agents as follows:

[TABLE]

Note that the condition $c_{i}^{5}(\mathbf{z})\leq 0$ for $i\in\mathcal{N}_{g}$ above corresponds to the constraint (43d). We modified this constraint without changing the problem (due to existence of the hard constraints (43b)) to be able to use the result from Theorem 1 requiring bounded gradients of the constraint functions. Given the properties of the cost and utility functions (see (37) and (38)), the objective function $F(\mathbf{z})=\sum_{i\in\mathcal{N}_{g}}F_{i}(\mathbf{z})+\sum_{j\in\mathcal{N}_{d}}F_{j}(\mathbf{z})$ is strongly convex and, hence, inf-compact. Moreover, Assumptions 1 and  2 hold for the gradients of the functions $F_{i}$, $F_{j}$ and local constraint functions $c_{i}^{1}$, $c_{i}^{2}$, $c_{i}^{3}$, $c_{i}^{4}$, $c_{i}^{5}$, $c_{j}^{1}$, $c_{j}^{2}$, $i\in\mathcal{N}_{g}$, $j\in\mathcal{N}_{d}$. Thus, the problem (43) is equivalent to

[TABLE]

and the following result can be formulated

Corollary 1**.**

Let Assumptions 4 and 5 hold for the problem (41). Then under an appropriate choice of the parameters $a_{t}$, $r_{t}$ (see Assumption 5), the penalty-based push-sum algorithm (6) applied to the reformulated problem (44) converges to the optimal solution to (41) as time tends to infinity.

III-B Simulation Results

In this section we will substantiate our theoretic result stated in Corollary 1 for the energy management problem (41) with simulations. For this purpose, we verify Theorem 1 by comparing the optimum $\bm{p}^{*}$ of the problem (41) with the iterations of the penalty-based push-sum algorithm presented in Section II.

For our simulation we use a small setup of two generator and two consumer nodes. The cost functions of the generators are as in (37), whereas the utility functions of the demand nodes are

[TABLE]

where $K_{j}<1$ is a positive constant for each $j\in\mathcal{N}_{d}$. Thus, the properties (38) are met. For the parameters for the cost functions, the lower and upper bounds on $p_{i}$ and $p_{j}$, as well as for the transmission loss coefficients $l_{i}$, $i\in\mathcal{N}_{g}$, we rely on settings in [17]. We model our time-varying communication architecture with a changing signal $s_{t}$ that chooses the current graph $G(s(t))$ sequentially from the set $\mathcal{G}=\left\{G_{1},G_{2}\right\}$, where $G_{1}$ and $G_{2}$ are not strongly connected but their union is. The communication architecture is depicted in Figure 1.

The results of our simulations are shown in Table I and Figure 2. The table states the optimal values $\bm{p}^{*}$ of the generators and consumers together with the result of the push-sum algorithm after $3\times 10^{4}$ iterations. The figure shows the convergence of the relative error $\mbox{error}_{k}=\frac{|p_{k}-p_{k}^{*}|}{p_{k}^{*}}$, $k\in[4]$, to zero as time runs. We can notice that the relative error at the demand nodes ($k=3,4$) approaches [math] already after $500$ iterations, whereas the generator’s errors need significantly more time to get close to [math]. This effect is due to a more complex structure of generators’ constraints, for which an optimal choice of penalty parameters needs to be studied in the future work.

IV Conclusion

In this paper we extended the distributed push-sum algorithm to the case of constrained convex optimization. The penalty-based push-sum algorithm was presented and its convergence to a system’s optimum was proven. We demonstrated applicability of the proposed procedure to distributed energy management in smart grid. The future work will focus on such questions as the convergence rate of the penalty-based push-sum algorithm and its dependence on the communication topology as well as an optimal choice of penalty functions and penalty parameters.