Distributed Continuous-Time and Discrete-Time Optimization With Nonuniform Unbounded Convex Constraint Sets and Nonuniform Stepsizes

TL;DR

This paper develops distributed optimization algorithms for multi-agent systems with unbounded and nonuniform constraints and stepsizes, ensuring convergence without requiring strong connectivity or bounded gradients.

Contribution

It introduces novel continuous and discrete-time distributed algorithms accommodating nonuniform, unbounded convex constraints and stepsizes, extending the scope of multi-agent optimization.

Findings

Agents reach consensus while minimizing the objective.

Algorithms work with unbounded, nonuniform constraints and stepsizes.

Numerical examples confirm theoretical convergence results.

Abstract

This paper is devoted to distributed continuous-time and discrete-time optimization problems with nonuniform convex constraint sets and nonuniform stepsizes for general differentiable convex objective functions. The communication graphs are not required to be strongly connected at any time, the gradients of the local objective functions are not required to be bounded when their independent variables tend to infinity, and the constraint sets are not required to be bounded. For continuous-time multi-agent systems, a distributed continuous algorithm is first introduced where the stepsizes and the convex constraint sets are both nonuniform. It is shown that all agents reach a consensus while minimizing the team objective function even when the constraint sets are unbounded. After that, the obtained results are extended to discrete-time multi-agent systems and then the case where each agent remains in a corresponding convex constraint set is studied. To ensure all agents to remain in a bounded region, a switching mechanism is introduced in the algorithms. It is shown that the distributed optimization problems can be solved, even though the discretization of the algorithms might deviate the convergence of the agents from the minimum of the objective functions. Finally, numerical examples are included to show the obtained theoretical results.

Tables

Table 1. TABLE I: Existing results on distributed optimization with nonuniform stepsizes.

distributed optimization	nonuniform stepsizes           (A)	nonuniform stepsizes           (B)
nonuniform convex constraints	unsolved	unsolved
jointly strongly connected graphs	unsolved	unsolved
unbounded gradients	partly solved with certain assumptions, e.g. [30, 31]	partly solved in [30] with discontinuous algorithms

Equations

$$\displaystyle\begin{array}[]{lll}\mathrm{minimize}~{}~{}\sum_{i=1}^{n}f_{i}(s)\\ \mathrm{subject~{}to}~{}~{}s\in H=\bigcap_{i=1}^{n}{H}_{i}\subseteq\mathbb{R}^{r},\end{array}$$

$$\displaystyle\begin{array}[]{lll}\mathrm{minimize}~{}~{}\sum_{i=1}^{n}f_{i}(x_{i})\\ \mathrm{subject~{}to}~{}~{}x_{i}=x_{j}\in H=\bigcap_{i=1}^{n}{H}_{i}\subseteq\mathbb{R}^{r}.\end{array}$$

$${\dot{x}}_i(t)=u_i(t),i\in \mathcal{I},$$

$$\displaystyle\begin{array}[]{lll}\dot{q}_{i}(t)&=&\mathrm{arctan}(e^{\|x_{i}(t)\|}),q_{i}(0)>0,\\ u_{i}(t)&=&\sum_{j\in N_{i}(t)}a_{ij}(t){\big{(}}x_{j}(t)-x_{i}(t){\big{)}}\\ &-&[x_{i}(t)-P_{H_{i}}(x_{i}(t))]-\frac{\nabla f_{i}(x_{i}(t))}{\sqrt{q_{i}(t)}}\end{array}$$

$$x(t)=[x_1(t{)}^T,\cdots ,x_n^T(t){]}^T,$$

$$Q(t)=\mathrm{diag}\{1/\sqrt{q_1(t)},\cdots ,1/\sqrt{q_n(t)}\},$$

$$\nabla f(x(t))=[\nabla f_1(x_1(t){)}^T,\cdots ,\nabla f_n(x_n(t){)}^T{]}^T$$

$$\displaystyle\begin{array}[]{lll}\dot{x}(t)=-L(t){x}(t)-Pz(x(t))-Q(t)\nabla f(x(t)).\end{array}$$

$$x_i((k+1)T)=u_i(kT),i\in \mathcal{I},$$

$$\displaystyle\begin{array}[]{lll}&{q}_{i}(k+1)={q}_{i}(k)+\mathrm{arctan}(e^{\|x_{i}(k)\|})T,q_{i}(0)>0,\\ &u_{i}(k)=w_{i}(k)(1-\gamma_{i})+P_{H_{i}}[w_{i}(k)]\gamma_{i},\\ &w_{i}(k)=v_{i}(k)-gr_{i}(k)T,\\ &v_{i}(k)=\sum_{j\in N_{i}(k)\cup\{i\}}a_{ij}(k)x_{j}(k),\\ &gr_{i}(k)=\left\{\begin{array}[]{lll}0,\mbox{~{}if~{}}\sqrt{q_{i}(k)}\leq\|\nabla f_{i}(v_{i}(k))\|^{2},\\ \frac{\nabla f_{i}(v_{i}(k))}{\sqrt{q_{i}(k)}},\mbox{~{}otherwise,}\end{array}\right.\end{array}$$

$$\underset{t\to +\infty }{\lim }{\Delta }_i(t)/q_1(t)=\underset{t\to +\infty }{\lim }[{\Delta }_i(T_1)$$

$$\displaystyle\begin{array}[]{lll}&\dot{V}_{0}(t)\\ &={\big{(}}x(t)-z\mathbf{1}{\big{)}}^{T}[-L(t){x}(t)-Pz(x(t))-Q(t)\nabla f(x(t))]\end{array}$$

$$\displaystyle\begin{array}[]{lll}-(x_{i}(t)-z)^{T}(x_{i}(t)-P_{H_{i}}(x_{i}(t)))\\ \leq\frac{1}{2}\|z-P_{H_{i}}(z)\|^{2}-\frac{1}{2}\|x_{i}(t)-P_{H_{i}}(x_{i}(t))\|^{2}\\ =-\frac{1}{2}\|x_{i}(t)-P_{H_{i}}(x_{i}(t))\|^{2}\end{array}$$

$$\displaystyle\begin{array}[]{lll}\dot{V}_{0}(t)\leq-\frac{1}{2}\sum_{i=1}^{n}\|x_{i}(t)-P_{H_{i}}(x_{i}(t))\|^{2}\\ -{\big{(}}x(t)-z\mathbf{1}{\big{)}}^{T}Q(t)\nabla f(x(t))\end{array}$$

$$\displaystyle\begin{array}[]{lll}\nabla f_{i}(x_{i}(t))^{T}(z-x_{i}(t))\leq f_{i}(z)-f_{i}(x_{i}(t))\leq f_{i}(z)-f_{i}(z_{i}).\end{array}$$

$$\displaystyle\begin{array}[]{lll}-{\big{(}}x(t)-z\mathbf{1}{\big{)}}^{T}Q(t)\nabla f(x(t))\\ \leq\sum_{i=1}^{n}\frac{1}{\sqrt{q_{i}(t)}}[f_{i}(z)-f_{i}(x_{i}(t))].\end{array}$$

$$\displaystyle\begin{array}[]{lll}-{\big{(}}x(t)-z\mathbf{1}{\big{)}}^{T}Q(t)\nabla f(x(t))\\ \leq-\frac{1}{\sqrt{q_{i_{0}}(t)}}[f_{i_{0}}(x_{i_{0}}(t))-f_{i_{0}}(z)]\\ +\sum_{j=1,j\neq i_{0}}^{n}\frac{1}{\sqrt{q_{j}(t)}}[f_{j}(z)-f_{j}(z_{j})]\\ \leq-\frac{1}{2\sqrt{t}}[f_{i_{0}}(x_{i_{0}}(t))-f_{i_{0}}(z)]\\ +\sum_{j=1,j\neq i_{0}}^{n}\frac{2}{\sqrt{t}}[f_{j}(z)-f_{j}(z_{j})]\leq 0.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\|x_{i}(t)-P_{H}(x^{*}(t))\|\|\nabla f_{i}(x_{i}(t))\|/\sqrt{q_{i}(t)}<\frac{\rho}{n\sqrt{t}}\end{array}$$

$$\displaystyle\begin{array}[]{lll}\dot{x}^{*}(t)=-\frac{1}{n}\sum_{i=1}^{n}[x_{i}(t)-P_{H_{i}}(x_{i}(t))+\frac{\nabla f_{i}(x_{i}(t))}{\sqrt{q_{i}(t)}}].\end{array}$$

$$V_1(t)=\sum _{i=1}^n\Vert x_i(t)-x^{\ast }(t){\Vert }^2+n\Vert x^{\ast }(t)-P_H(x^{\ast }(t)){\Vert }^2$$

$$\displaystyle\begin{array}[]{lll}&\dot{V}_{1}(t)\\ =&2\sum_{i=1}^{n}[x_{i}(t)-x^{*}(t)]^{T}(\dot{x}_{i}(t)-\dot{x}^{*}(t))\\ +&2n[x^{*}(t)-P_{H}(x^{*}(t)]^{T}\dot{x}^{*}(t)\\ =&-2x^{T}(t)L(t)x(t)-2\sum_{i=1}^{n}[x_{i}(t)-x^{*}(t)]^{T}\\ \times&[(x_{i}(t)-P_{H_{i}}(x_{i}(t))+\frac{\nabla f_{i}(x_{i}(t))}{\sqrt{q_{i}(t)}}]\\ -&2\sum_{i=1}^{n}[x^{*}(t)-x_{i}(t)+x_{i}(t)-P_{H}(x^{*}(t))]^{T}\\ \times&[x_{i}(t)-P_{H_{i}}(x_{i}(t))+\frac{\nabla f_{i}(x_{i}(t))}{\sqrt{q_{i}(t)}}]\\ =&-2x^{T}(t)L(t)x(t)-2\sum_{i=1}^{n}[x_{i}(t)-P_{H}(x^{*}(t))]^{T}\\ \times&[x_{i}(t)-P_{H_{i}}(x_{i}(t))+\frac{\nabla f_{i}(x_{i}(t))}{\sqrt{q_{i}(t)}}]\\ \leq&-\phi(t)+\frac{2\rho}{\sqrt{t}}\end{array}$$

$$\displaystyle\begin{array}[]{lll}\phi(t)\triangleq 2x(t)^{T}L(t){x}(t)+\sum_{i=1}^{n}\|x_{i}(t)-P_{H_{i}}(x_{i}(t))\|^{2},\end{array}$$

$$\displaystyle\begin{array}[]{lll}\int_{t_{k}}^{t_{k+1}}\phi(s)\mathrm{d}s\geq h_{1}(t_{k})^{2}/h_{0}/2\end{array}$$

$$\displaystyle\begin{array}[]{lll}\int_{t_{k}}^{t_{k+1}}\phi(s)\mathrm{d}s\geq(2h_{1}(t_{k})-h_{0}d_{w}/2)d_{w}/4\geq d_{w}^{2}h_{0}/8.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\int_{t_{k}}^{t_{k+1}}\phi(s)\mathrm{d}s\geq\min\{h_{1}(t_{k})^{2}/h_{0}/2,d^{2}_{w}h_{0}/8\}\geq\epsilon(M+1).\end{array}$$

$$\displaystyle\begin{array}[]{lll}V_{1}(t_{k+1})-V_{1}(t_{k})\leq\int_{t_{k}}^{t_{k+1}}[-\phi(s)+\epsilon]\mathrm{d}s\leq-\epsilon.\end{array}$$

$$\displaystyle\begin{array}[]{lll}x^{T}(t)L(t)x(t)\\ =\sum_{i=1}^{n}\sum_{j\in N_{i}(t)}a_{ij}(t)\|x_{i}(t)-x_{j}(t)\|^{2}\\ \geq\sum_{i=1}^{n}\sum_{j\in N_{i}(t)}\eta\|x_{i}(t)-x_{j}(t)\|^{2}.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\max_{i}\{\|x^{*}(t)-x_{i}(t)\|\mid t\in[{t_{k}},{t_{k+1}}]\}\\ \leq\max_{i,j}\{\|x_{i}(t)-x_{j}(t)\|\mid t\in[{t_{k}},{t_{k+1}}]\}\\ \leq 2n[\sqrt[4]{2h_{0}(M+1)\epsilon}(1+\bar{\eta}n/\sqrt{\eta})+\epsilon]M\\ +n\sqrt[4]{2h_{0}(M+1)\epsilon}/\sqrt{\eta}.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\phi_{1}(t)\triangleq\sum_{i=1}^{n}\|x_{i}(t)-x^{*}(t)\|^{2}\\ +\sum_{i=1}^{n}\|x^{*}(t)-P_{H_{i}}(x^{*}(t))\|^{2}.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\|x^{*}(t)-P_{H_{i}}(x^{*}(t))\|\\ =\|x^{*}(t)-x_{i}(t)+x_{i}(t)-P_{H_{i}}(x_{i}(t))\\ +P_{H_{i}}(x_{i}(t))-P_{H_{i}}(x^{*}(t))\|\\ \leq 2\|x^{*}(t)-x_{i}(t)\|+\|x_{i}(t)-P_{H_{i}}(x_{i}(t))\|.\end{array}$$

Full text

Keywords: Distributed Optimization, Nonuniform Step-Sizes, Nonuniform Convex Constraint Sets

I Introduction

As an important research direction of control theory, distributed optimization problems for multi-agent systems have attracted more and more attention from the control community [1, 2, 3, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 4, 5, 16, 17, 18, 19, 20, 21, 22, 24, 25, 23, 26, 27, 28, 29, 31, 30]. The goal of a distributed optimization problem for a multi-agent system is to minimize a desired team objective function cooperatively in a distributed way where each agent can only have access to partial information of the team objective function. During the past few years, several results have been obtained for distributed optimization problems. For example, article [1] introduced a discrete-time projection algorithm for multi-agent systems with state constraints and proved that the optimization problems can be solved when the communication topologies are jointly strongly connected and balanced. Articles [4] and [5] studied a continuous-time version of the work in [1] with convex constraint sets. Article [6] gave a distributed continuous-time dynamic algorithm for distributed optimization, and subsequently, on this basis, articles [7, 8] studied the distributed optimization problem for general strongly connected balanced directed graphs and gave the estimate of the convergence rate of the algorithm. Other works about distributed optimization problems could be found in articles [2, 3, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 25, 23, 26, 27, 28, 29, 31, 30] and the references therein, where new algorithms, e.g., distributed Newton, approximate dual subgradient and zero-gradient-sum algorithms, were given or more complicated cases, e.g., second-order dynamics, time-varying or nonconvex functions, fixed or asynchronous stepsizes and noise, were considered.

Though many excellent results have been obtained for the distributed optimization problem, many issues need be further studied, e.g., general convex functions, nonuniform convex constraints and nonuniform stepsizes. For the issues of general convex functions and nonuniform convex constraints, most of the existing results require the gradients or subgradients of the convex functions to be bounded and the convex constraints to be identical and little attention has been paid to general convex functions and nonuniform convex constraints, in particular for multi-agent systems with general directed balanced graphs and unbounded gradients. For example, articles [7, 8] studied general convex functions but assumed them to be globally Lipschitz and the graphs are assumed to be strongly connected and balanced. Article [5] studied coercive convex functions with unbounded subgradients but the results are limited to the continuous-time multi-agent systems and the convex constraints sets are assumed to be identical for all agents. Article [1] studied nonuniform convex constraints but the communication graph is complete and all the edge weights are assumed to be equal. Founded on [1], articles [24, 25] gave some results on nonuniform convex constraints but the communication graph is constant and connected, and the objective functions are assumed to be strongly convex or some intermediate variables need be transmitted besides the agent states. Article [29] studied a distributed optimization problem with nonuniform convex constraints and gave conditions to guarantee the optimal convergence of the team objective function, but the subgradients and the convex constraint sets are both bounded. Note that [1, 5, 24, 25, 29] all adopt a uniform stepsize. For the issue of nonuniform stepsizes, currently, there are few works concerned about this issue. Articles [10, 11] studied the distributed optimization problem with nonuniform stepsizes in a stochastic setting, where the communication graphs are required to be undirected and connected. Article [30] introduced a kind of nonuniform stepsizes but the discontinuous algorithms were employed to realize the consensus of all agents and the convex constraint sets are assumed to be identical. Article [31] also studied the distributed optimization problem with nonuniform stepsizes but some intermediate variables need be transmitted besides the agent states in order to track the average of the gradients.

In this paper, we are interested in studying distributed continuous-time and discrete-time optimization problems with nonuniform convex constraint sets and nonuniform stepsizes for general differentiable convex objective functions. The communication graphs might not be strongly connected at any time and it is only required that the union of the communication graphs among the time intervals of a certain length be strongly connected. The gradients of the local objective functions considered might not be bounded when their independent variables tend to infinity. First, a distributed continuous-time algorithm is introduced where the stepsizes and the convex constraint sets are both nonuniform. Nonuniform stepsizes mean that the weights of the gradients of the local objective functions in the control input of the agents are nonuniform. That is, the optimal convergence rates of the local objective functions are different, which has great possibility to result in the destruction of the optimal convergence of the team objective function. The existing works (e.g., [1]) usually assumed the stepsizes are uniform and took no consideration of nonuniform stepsizes, and hence their approaches are hard to be applied for the case of nonuniform stepsizes. Our approach is to introduce a kind of stepsizes such that the stepsizes of each agent are constructed only based on its own states and the differences between the stepsizes of all agents vanish to zero as time evolves. Though the stepsizes of all agents tend to the same as time evolves, their differences still heavily affect the consensus stability and optimal convergence of the system, especially when the communication graphs are not strongly connected. Moreover, due to the existence of nonuniform convex constraint sets, we need take into account the nonlinearity of the consensus and optimal convergence caused by nonuniform convex constraint sets, which renders the analysis of this case to be very complicated. In particular when the nonuniform constraint sets are unbounded, the gradients of the local objective functions might tend to be unbounded when their independent variables tend to infinity, which makes the existing approaches invalid, e.g., [1], [29], where the nonuniform constraint sets and the subgradients were both bounded. To solve the optimization problem, we perform the analysis in three steps. The first step is to make full use of the convexity of the objective functions and show that our algorithm ensures that all agents remain in a bounded region for all the time. The second step is to analyze the agent dynamics at some key times and show the consensus convergence of all agents. The third step is to estimate the consensus errors and use the estimation of the distance from the agents to the convex constraint sets and the convexity of the objective functions to show the optimal convergence of the optimization problem. After that, we extend the obtained results to discrete-time multi-agent systems, and then study the case where each agent remains in its corresponding convex constraint set. Due to the discretization of the system dynamics, the agents might deviate from the minimum of the objective functions. Such a problem also exists in the centralized optimization system. To deal with it, a switching mechanism is introduced in the algorithms based on each agent’s own information under which all agents remain in a bounded region even when the convex constraint sets are unbounded. It is shown that the distributed optimization problems with nonuniform convex constraint sets and nonuniform stepsizes can be solved for the discrete-time multi-agent systems.

This paper takes nonuniform unbounded convex constraint sets, nonuniform stepsizes, general differentiable convex objective functions, general switching graphs and the discretization of the algorithms into account simultaneously for the distributed optimization problems. The nonlinearities caused by these factors are different and the coexistence of these nonlinearities would further result in more complicated nonlinearities. Existing works only addressed a fraction of these factors due to the limitations of the algorithms and the analytical approaches. For example, the algorithms in [7, 8] cannot be directly applied to the case of convex constraint sets due to the adoption of the integrator operator. The analytical approaches in [30, 31] cannot be directly applied in this paper, because the nonsmooth sign functions are used in [30] to account for inconsistency in gradients while some intermediate variables need be transimitted besides the agent states in [31]. Neither feature is valid in the current paper as continuous functions are used and no intermediate variables are transmitted. Moreover, in [30, 31], the communication graphs are assumed to be strongly connected at all time or the constraint sets are assumed to be identical, which makes the analytical approaches in [30, 31] unable to be directly applied in this paper as well.

Notation: $\mathbb{R}^{m}$ denotes the set of all $m$ dimensional real column vectors; $\mathbb{R}^{m\times n}$ denotes the set of all $m\times n$ real matrices; $\mathcal{I}$ denotes the index set $\{1,\ldots,n\}$; 1 represents a column vector of all ones with a compatible dimension; $s_{i}$ denotes the $i$th component of the vector $s$; $A_{ij}$ denotes the $(i,j)$th entry of the matrix $A$; $s^{T}$ and $A^{T}$ denote, respectively, the transpose of the vector $s$ and the matrix $A$; $||s||$ denotes the Euclidean norm of the vector $s$; $\nabla f(s)$ denotes the gradient of the function $f(s)$ at $s$; $\mathrm{diag}\{A_{1},\cdots,A_{n}\}$ denotes a block diagonal matrices with its diagonal blocks equal to the matrices $A_{i}(k)$; the symbol $/$ denotes the division sign; and $P_{X}(s)$ denotes the projection of the vector $s$ onto the closed convex set $X$, i.e., $P_{X}(s)=\mathrm{arg}\min\limits_{\bar{s}\in X}\|s-\bar{s}\|$.

II Preliminaries and problem formulation

II-A Preliminaries

Let $\mathcal{G}(\mathcal{I},\mathcal{E},\mathcal{A})$ be a directed communication graph of agents, where $\mathcal{E}\subseteq\mathcal{I}\times\mathcal{I}$ is the set of edges, and $\mathcal{A}=[a_{ij}]\in\mathbb{R}^{n\times n}$ is the weighted adjacency matrix. An edge $(i,j)\in\mathcal{E}$ denotes that agent $j$ can obtain information from agent $i$. The weighted adjacency matrix $\mathcal{A}$ is defined as $\eta\leq a_{ij}\leq\bar{\eta}$ for two constants $\bar{\eta}>\eta>0$ if $(j,i)\in\mathcal{E}$ and $a_{ij}=0$ otherwise. It is assumed by default that $a_{ii}=0$, i.e., $(i,i)\notin\mathcal{E}$. The Laplacian of the graph $\mathcal{G}$, denoted by $L$, is defined as $L_{ii}=\sum_{j=1,j\neq i}^{n}a_{ij}$ and $L_{ij}=-a_{ij}$ for all $i\neq j$. The graph $\mathcal{G}$ is undirected if $a_{ij}=a_{ji}$ for all $i,j$, and it is balanced if $\sum_{j=1}^{n}a_{ij}=\sum_{j=1}^{n}a_{ji}$ for all $i$. The set of neighbors of agent $i$ is denoted by $N_{i}=\{j\in\mathcal{I}\mid(j,i)\in\mathcal{E}\}.$ A path is a sequence of edges of the form $(i_{1},i_{2}),(i_{2},i_{3}),\cdots$, where $i_{j}\in\mathcal{I}$. The graph $\mathcal{G}$ is strongly connected, if there is a path from every agent to every other agent, and the graph $\mathcal{G}$ is connected, if it is undirected and strongly connected [33].

Lemma 1**.**

[33] If the graph $\mathcal{G}$ is strongly connected, the Laplacian $L$ has one zero eigenvalue associated with eigenvalue vector $\mathbf{1}$ and all its rest $n-1$ eigenvalues have positive real parts. Further, if the graph $\mathcal{G}$ is undirected and connected, all the $n-1$ nonzero eigenvalues are positive.

Lemma 2**.**

[34] Let $f_{0}(\chi):\mathbb{R}^{r}\rightarrow\mathbb{R}$ be a differentiable convex function. $f_{0}(\chi)$ is minimized if and only if $\nabla f_{0}(\chi)=0$.

Lemma 3**.**

[35] Suppose that $Y\neq\emptyset$ is a closed convex set in $\mathbb{R}^{r}$. The following statements hold.

(1) For any $y\in\mathbb{R}^{r}$, $\|y-P_{Y}(y)\|$ is continuous with respect to $y$ and $\nabla\frac{1}{2}\|y-P_{Y}(y)\|^{2}=y-P_{Y}(y)$;

(2) For any $y,z\in\mathbb{R}^{r}$ and all $\xi\in Y$, $[y-P_{Y}(y)]^{T}(y-\xi)\geq 0$, $\|P_{Y}(y)-\xi\|^{2}\leq\|y-\xi\|^{2}-\|P_{Y}(y)-y\|^{2}$ and $\|P_{Y}(y)-P_{Y}(z)\|\leq\|y-z\|.$

II-B Problem formulation

Consider a multi-agent system consisting of $n$ agents. Each agent is regarded as a node in a directed graph $\mathcal{G}$111In the following, $\mathcal{G}(t)$, $\mathcal{G}(kT)$, $a_{ij}(t)$, $a_{ij}(kT)$, $N_{i}(t)$ and $N_{i}(kT)$ will be used to denote the graph, the edge weight and the agent neighbor set at time $t$ or $k$ as defined in Sec. II.A., and each agent can interact with only its local neighbors in $\mathcal{G}$. Our objective is to design algorithms using only local interaction and information such that all agents cooperatively find the optimal state that solves the optimization problem

[TABLE]

where $f_{i}(s):\mathbb{R}^{r}\rightarrow\mathbb{R}$ denotes the differentiable convex local objective function of agent $i$, and $H_{i}$ denotes the closed convex constraint set of $f_{i}(s)$. Clearly, $\sum_{i=1}^{n}f_{i}(s)$ is also a differentiable convex function. It is assumed that $f_{i}(s)$ and $H_{i}$ are known only to agent $i$. The problem described above is equivalent to the problem that all agents reach a consensus while minimizing the team objective function $\sum_{i=1}^{n}f_{i}(x_{i})$, i.e.,

[TABLE]

In this paper, our analysis is for the general $m$ case. When no confusion arises, the equations or formula are written in the form of $m=1$ for notational simplicity.

III Distributed continuous-time optimization

In this section, we discuss the distributed optimization problem for continuous-time multi-agent systems. The problem has applications in motion coordination of multi-agent systems, where multiple physical vehicles rendezvous or form a formation centered at a team optimal location. Suppose that the agents satisfy the continuous-time dynamics

[TABLE]

where $x_{i}\in\mathbb{R}^{r}$ is the state of agent $i$, and $u_{i}\in\mathbb{R}^{r}$ is the control input of agent $i$.

III-A Assumptions and some necessary lemmas

Let $\mathcal{X}$ denote the optimal set of the constrained optimization problem (4). Before the main assumptions and the necessary lemmas, we need further define the sets X_{i}\triangleq{\Big{\{}}x{\Big{|}}\nabla f_{i}(x)=0{\Big{\}}} for all $i\in\mathcal{I}$ and X\triangleq{\Big{\{}}x{\Big{|}}\sum_{i=1}^{n}\nabla f_{i}(x)=0{\Big{\}}} for later usage. From Lemma 2, $X_{i}$ and $X$ are convex and denote, respectively, the optimal sets of $f_{i}(x)$ and the team objective function $\sum_{i=1}^{n}f_{i}(x)$ for $x\in\mathbb{R}^{r}$. Note that in general $X$ is different from $\mathcal{X}$ but $X=\mathcal{X}$ when $H_{i}=\mathrm{R}^{r}$ for all $i$.

Assumption 1**.**

[30] Each set $X_{i}$, $i\in\mathcal{I}$, is nonempty and bounded.

In Assumption 1, we only make an assumption on each $f_{i}(x)$ rather than the team objective function $\sum_{i=1}^{n}f_{i}(x)$ because $\sum_{i=1}^{n}f_{i}(x)$ is global information for all agents and cannot be used by each agent in a distributed way.

Assumption 2**.**

$H\neq\emptyset$.

In Assumption 2, we do not require $H_{i}$ to be bounded and hence $H$ might be unbounded.

Lemma 4**.**

[30] Under Assumption 1, the following two statements hold:

(1) $\lim_{\|y\|\rightarrow+\infty}f_{i}(y)=+\infty$ for all $i$ and accordingly $\lim_{\|y\|\rightarrow+\infty}\sum_{i=1}^{n}f_{i}(y)=+\infty$.

(2) All $X_{i}$, $i\in\mathcal{I}$, and $X$ are nonempty closed bounded convex sets.

Lemma 5**.**

[30] Under Assumptions 1 and 2, $\mathcal{X}$ is a nonempty closed bounded convex set.

Actually, Lemma 5 shows the existence and boundedness of the optimal set of the constrained optimization problem (4), $\mathcal{X}$, under Assumptions 1 and 2.

Assumption 3**.**

[30] The length of the time interval between two contiguous switching times is no smaller than a given constant, denoted by $d_{w}$.

Under Assumption 3, the switching of the graph $\mathcal{G}(t)$ cannot be arbitrarily fast, which prevents the system from exhibiting the Zeno behavior.

Assumption 4**.**

There exists an infinite sequence of swiching times of the graph $\mathcal{G}(t)$, $t_{0},t_{1},t_{2},\cdots,$ such that $t_{0}=0,$ $0<t_{j+1}-t_{j}\leq M$ and the union of all the graphs during each interval $[t_{j},t_{j+1})$ is strongly connected for some constant $M>d_{w}$ and all nonnegative integers $j$.

Assumption 4 ensures that all agents can communicate with each other persistently. Suppose that the graph $\mathcal{G}(t)$ is balanced for all $t$. From [32], by rearranging the agent indices, $\frac{L(t)^{T}+L(t)}{2}$ can be written as $\frac{L(t)^{T}+L(t)}{2}=\mathrm{diag}\{\frac{L^{1}(t)^{T}+L^{1}(t)}{2},\cdots,\frac{L^{h}(t)^{T}+L^{h}(t)}{2}\}$, where each $\frac{L^{i}(t)^{T}+L^{i}(t)}{2}$ corresponds to a strongly connected component of the agents. From Lemma 1, each eigenvalue of $\frac{L^{i}(t)^{T}+L^{i}(t)}{2}$ is nonnegative and hence all eigenvalues of $\frac{L(t)^{T}+L(t)}{2}$ are nonnegative.

Before the main results, we first present some necessary lemmas that will be used in the analysis of the main results. Specifically, Lemma 6 shows a radial growth property of the derivatives of $f_{i}(x)$, Lemma 7 shows the consensus convergence property of the stepsizes, Lemma 8 shows a dependency relationship between the distances from one given point to the convex sets and their intersection, and Lemma 9 shows the boundedness of the gradients in a bounded region. For clarity, the proofs of Lemmas 7-9 together with Theorems 1-4 are provided in the Appendix.

Lemma 6**.**

[30] Let $f(s):\Xi\mapsto\mathbb{R}$ be a differentiable convex function and $Y$ be its minimum set in $\Xi$, where $\Xi\subseteq\mathbb{R}^{r}$ is a closed convex set. Suppose that $Y\subseteq\Xi$ is closed and bounded. For any $z=\lambda P_{Y}(y)+(1-\lambda)y$ with $\lambda\in(0,1)$, $0<\nabla f(z)^{T}\frac{y-P_{Y}(y)}{\|y-P_{Y}(y)\|}\leq\nabla f(y)^{T}\frac{y-P_{Y}(y)}{\|y-P_{Y}(y)\|}$ for any $y\in\Xi-Y$.

Lemma 7**.**

For the system given by $\dot{q}_{i}(t)=\mathrm{arctan}(e^{\|x_{i}(t)\|})$ with $q_{i}(0)>0$, if $\lim_{t\rightarrow+\infty}[x_{i}(t)-x^{*}(t)]=0$ for all $i$, $\lim_{t\rightarrow+\infty}\frac{q_{i}(t)}{q_{j}(t)}=1$ for all $i,j$, where $x^{*}(t)\triangleq\frac{1}{n}\sum_{i=1}^{n}x_{i}(t)$.

Lemma 8**.**

Let $y(t)\in E$ for all $t$, where $E$ is a bounded set. Under Assumption 2, if $\lim_{t\rightarrow+\infty}\|y(t)-P_{H_{i}}(y(t))\|=0$ for all $i$, then $\lim_{t\rightarrow+\infty}\|y(t)-P_{H}(y(t))\|=0$.

Lemma 9**.**

Let $Y$ be a closed bounded convex set. Then, $\|\nabla f_{i}(x)\|<\varrho$ for all $i$, all $x\in Y$ and some constant $\varrho>0$.

III-B Algorithm and convergence analysis

In this subsection, we design a continuous distributed optimization algorithms with nonuniform stepsizes. The algorithm is given by

[TABLE]

for all $i$. The stepsize of the gradient, $1/\sqrt{q_{i}(t)}$, is used to make the term $\frac{\nabla f_{i}(x_{i}(t))}{\sqrt{q_{i}(t)}}$ tend to zero as $t\rightarrow+\infty$. The role of the term $-\frac{\nabla f_{i}(x_{i}(t))}{\sqrt{q_{i}(t)}}$ is to make all agents converge to the optimal set of the team objective function and the role of $-[x_{i}(t)-P_{H_{i}}(x_{i}(t))]$ is to make each agent converge to the convex set $H_{i}$.

It should be noted that the construction of the stepsize $1/\sqrt{q_{i}(t)}$ is only based on the $i$th agent’s own states and it does not use the Lipschitz constant or the convexity constant as in the existing works, e.g., [7] and [24]. The stepsize of each agent $1/\sqrt{q_{i}(t)}$ is state-dependent and can be nonuniform for all agents. The existing works often assume the stepsizes of all agents to be predesigned and consistent with each other at any time, which exerts a heavy burden on sensing and communication costs of the entire system.

Remark 1**.**

In algorithm (9), the role of the inverse tangent functions and the exponential functions is to ensure $\dot{q}_{i}(t)$ to be upper and lower bounded. In fact, some other more general functions, e.g., saturation function, can be employed to play the same role. Moreover, the stepsizes used in algorithm (9) are in a special form, and it can also be extended to other functions. For easy readability, we do not give the general form of the functions $q_{i}(t)$ and the stepsizes.

Let

[TABLE]

[TABLE]

[TABLE]

and $Pz(x(t))=[x_{1}(t)-P_{H_{1}}(x_{1}(t)),\cdots,x_{n}(t)-P_{H_{n}}(x_{n}(t))]^{T}$. Then the system (5) with (9) can be written as

[TABLE]

For convenience of discussion, under Assumption 4, let $t_{j_{0}}<t_{j_{1}}<\cdots<t_{j_{m_{j}}}$ with $t_{j_{0}}=t_{j}$ and $t_{j_{m_{j}}}=t_{j+1}$ denote all the switching times in the interval $[t_{j},t_{j+1})$. In the following, we show the effectiveness of this algorithm to solve the optimization problem (4).

Theorem 1**.**

Suppose that the graph $\mathcal{G}(t)$ is balanced for all $t$ and Assumptions 1, 2, 3 and 4 hold. For arbitrary initial conditions $x_{i}(0)\in\mathbb{R}^{r}$, using algorithm (9) for system (5), the following statements hold.

• (1)

All $x_{i}(t)$ remain in a bounded region for all $i$ and all $t$.

• (2)

$\lim_{t\rightarrow+\infty}\|x^{*}(t)-P_{H}(x^{*}(t))\|=\lim_{t\rightarrow+\infty}[x_{i}(t)-x^{*}(t)]=0$ for all $i$, where $x^{*}(t)$ has been defined in Lemma 7.

Theorem 2**.**

Suppose that the graph $\mathcal{G}(t)$ is balanced for all $t$ and Assumptions 1, 2, 3 and 4 hold. For arbitrary initial conditions $x_{i}(0)\in\mathbb{R}^{r}$, using algorithm (9) for system (5), all agents reach a consensus, i.e., $\lim_{t\rightarrow+\infty}[x_{i}(t)-x^{*}(t)]=0$ for all $i$, and minimize the team objective function (4) as $t\rightarrow+\infty$. **

Remark 2**.**

In the existing works, the distributed optimization problems were considered often under the assumption that the stepsizes of the gradients are uniform and explicitly time-dependent for all agents. That is, the stepsizes should be consistent with each other at any time. In algorithm (9), we do not make such an assumption and the stepsizes of the gradients only depend on the agent states. The stepsizes need not have the same value at any time and instead they are usually nonuniform, which greatly relaxes the synchronization requirement on the system.

IV Distributed discrete-time optimization

In this section, we discuss the distributed optimization problem for discrete-time multi-agent systems. Suppose that the agents satisfy the discrete-time dynamics

[TABLE]

where $x_{i}\in\mathbb{R}^{r}$ is the state of agent $i$, $u_{i}\in\mathbb{R}^{r}$ is the control input of agent $i$, and $T>0$ is the sample time. In the following, we use “$(k)$” instead of “$(kT)$” when no confusion arises.

IV-A Assumptions and some necessary lemmas

In Sec. II, when we define the weighted adjacency matrix $\mathcal{A}$, we assume by default that $a_{ii}=0$. In this section, for discussion of discrete-time multi-agent systems, we need to redefine $a_{ii}$ and make an assumption about the weighted adjacency matrix $\mathcal{A}$ as shown in the following assumption.

Assumption 5**.**

[29] For all $i,j\in\mathcal{I}$, $a_{ii}(k)\geq\eta$, $a_{ij}(k)\geq\eta$ for some constant $0<\eta\leq 1$ and each nonzero $a_{ij}(k)$, $\sum_{i=1}^{n}a_{ij}(k)=1$ and $\sum_{j=1}^{n}a_{ji}(k)=1$.

Under Assumption 5, the adjacency matrix of the graph $\mathcal{G}(k)$ is doubly stochastic and its diagonal entries are nonzero. Assumption 5 is used to generate convex combinations of the agents’ states such that the influence of each agent’s state is equal in the final consensus value in the distributed optimization algorithms shown later.

Assumption 6**.**

[29] There exists an infinite sequence of switching times of the graph $\mathcal{G}(k)$, $k_{0},k_{1},k_{2},\cdots,$ such that $k_{0}=0$, $0<k_{j+1}-k_{j}\leq M$ and the union of all the graphs during each interval $[k_{j},k_{j+1})$ is strongly connected for some positive integer $M$ and all nonnegative integers $j$.

Similar to Assumption 4, Assumption 6 ensures that all agents can communicate with each other persistently.

IV-B Distributed optimization with nonuniform stepsizes

In this subsection, we design a discrete-time distributed optimization algorithm with nonuniform stepsizes. The algorithm is given by

[TABLE]

where $0<\gamma_{i}\leq 1$ is a constant for each $i$.

Due to the discretization of the system dynamics, the agents might deviate from the minimum of the team objective function. Such a problem also exists in the centralized optimization system. To deal with it, a switching mechanism is introduced in (20) based on each agent’ own information, under which the gradient term would not be too large to result in the divergence of the system. This will be shown in the proof of Theorem 3.

For Algorithm (20), it can be calculated simply in four steps: (a) $v_{i}(k)$, $q_{i}(k+1)$ and $\nabla f_{i}(v_{i}(k))$; (b) $gr_{i}(k)$ based on the switching mechanism; (c) $w_{i}(k)$; and (d) $P_{H_{i}}[w_{i}(k)]$ and $u_{i}(k)$. Though the algorithm computation looks a bit complex due to the existence of the switching mechanism, the algorithm does not require intermediate variables to be transmitted and it is a fully distributed algorithm.

Theorem 3**.**

Suppose that Assumptions 1, 2, 5 and 6 hold. For arbitrary initial conditions $x_{i}(0)\in\mathbb{R}^{r}$, using algorithm (20) for system (12), if $0<\gamma_{i}<1$ for all $i$, all agents reach a consensus, i.e., $\lim_{t\rightarrow+\infty}[x_{i}(k)-x^{*}(k)]=0$ for all $i$, and minimize the team objective function (4) as $k\rightarrow+\infty$. **

In Theorems 2 and 3, it is not required that each agent remain in its corresponding convex constraint set $H_{i}$. In the following theorem, we show that the optimization problem (4) can be solved when all agents remain in their corresponding convex constraint sets.

Theorem 4**.**

Suppose that Assumptions 1, 2, 5 and 6 hold. For arbitrary initial conditions $x_{i}(0)\in H_{i}$, using algorithm (20) for system (12), if $\gamma_{i}=1$ for all $i$, all agents reach a consensus, i.e., $\lim_{t\rightarrow+\infty}[x_{i}(k)-x^{*}(k)]=0$ for all $i$, and minimize the team objective function (4) as $k\rightarrow+\infty$ while each agent remains in its corresponding constraint sets, i.e., $x_{i}(k)\in H_{i}$ for all $i$ and all $k$. **

Remark 3**.**

Since the proposed algorithms are gradient based, the convergence rate of the algorithms is not very fast. This is common for the gradient-based distributed algorithms in the existing literature. In particular, the stepsizes (gradient gains) are nonuniform, which makes the convergence rate slower than that with uniform stepsizes. However, our algorithms are able to deal with the general case of nonuniform stepsizes without intermediate variables being transmitted. In the existing non-gradient-based works, some special assumptions are always made in order to ensure the optimal convergence. For example, in [9], the communication graphs are assumed to be strongly connected and the the convex objective functions are assumed to be strongly convex. In this paper, the communication graphs are only required to be jointly strongly connected, and the convex objective functions are only required to be differentiable (which can be easily extended to the nondifferentiable case by using subgradients). Future work could be directed towards improving the convergence rate of our algorithms. In particular, different dimension might yield different convergence rates. It is worth studying the effects of different dimensions on the convergence rate of the algorithms.

V Simulations

Consider a multi-agent system with 8 agents in $\mathbb{R}^{2}$. The communication graphs switch among the balanced subgraphs of the graph shown in Fig. 1. Each edge weight is 0.5. The sample time is $T=0.1$ $s$ for the discrete-time algorithm. The local objective functions are adopted as $f_{1}(x_{1})=\frac{1}{2}x_{11}^{2}+\frac{1}{2}x_{12}^{2},$ $f_{2}(x_{2})=\frac{1}{2}(x_{21}+1)^{2}+\frac{1}{2}x_{22}^{2},$ $f_{3}(x_{3})=\frac{1}{2}x_{31}^{2}+\frac{1}{2}(x_{32}+1)^{2},$ $f_{4}(x_{4})=\frac{1}{2}(x_{41}+1)^{2}+\frac{1}{2}(x_{42}+1)^{2}$, $f_{5}(x_{5})=\frac{1}{4}x_{51}^{4}+\frac{1}{4}x_{52}^{4},$ $f_{6}(x_{6})=\frac{1}{4}(x_{61}+1)^{4}+\frac{1}{4}x_{62}^{4},$ $f_{7}(x_{7})=\frac{1}{4}x_{71}^{4}+\frac{1}{4}(x_{72}+1)^{4}$, and $f_{8}(x_{8})=\frac{1}{4}(x_{81}+1)^{4}+\frac{1}{4}(x_{82}+1)^{4}$. where $x_{i1}$ and $x_{i2}$ denote the $1$st and $2$nd components of $x_{i}$. The constrained convex sets are adopted as $H_{1}=\{(x,y)^{T}\in\mathbb{R}^{2}\mid\|s-[0,0]^{T}\|\leq 3\}$ for agents 1 and 5, $H_{2}=\{(x,y)^{T}\in\mathbb{R}^{2}\mid\|x\leq 0.5,y\geq 1\}$ for agents 2 and 6, $H_{3}=\{(x,y)^{T}\in\mathbb{R}^{2}\mid\|s-[0,3]^{T}\|\leq 3\}$ for agents 3 and 7 and $H_{4}=\{(x,y)^{T}\in\mathbb{R}^{2}\mid x\geq-0.5,y\geq 1\}$ for agents 4 and 8. The team objective function $\sum_{i=1}^{8}f_{i}(s)$ is minimized if and only if $s=[-0.5,1]^{T}$. The simulation results for algorithm (9) and algorithm (20) with $\gamma_{i}=1$ for all $i$ are shown in Figs. 2 and 3. It is clear that all agents finally converge to the optimal point. In particular, for algorithm (20), all agents remain in their corresponding constraint sets. All the simulation results are consistent with our obtained theorems.

VI Conclusions

In this paper, distributed continuous-time and discrete-time optimization problems with nonuniform stepsizes and nonuniform possibly unbounded convex constraint sets were studied for general differentiable convex objective functions. One continuous-time algorithm and one discrete-time algorithm were introduced. For these two algorithms, it has been shown that the team objective function is minimized with nonuniform stepsizes and nonuniform convex constraint sets. In particular, for the discrete-time algorithm, it has been shown that the distributed optimization problem can be solved when each agent remain in its corresponding constraint set.