Distributed optimization with nonconvex velocity constraints, nonuniform position constraints and nonuniform stepsizes

TL;DR

This paper introduces distributed algorithms for multi-agent systems with complex constraints, ensuring bounded positions and convergence under nonconvex velocity and position constraints with nonuniform stepsizes.

Contribution

It proposes novel distributed algorithms that handle nonconvex velocity constraints, nonuniform position constraints, and adaptive stepsizes without predesign, ensuring boundedness and convergence.

Findings

Algorithms guarantee agents' positions remain bounded.

Convergence achieved under jointly strongly connected and balanced communication topologies.

Numerical examples validate theoretical results.

Abstract

This note is devoted to the distributed optimization problem of multi-agent systems with nonconvex velocity constraints, nonuniform position constraints and nonuniform stepsizes. Two distributed constrained algorithms with nonconvex velocity constraints and nonuniform stepsizes are proposed in the absence and the presence of nonuniform position constraints by introducing a switching mechanism to guarantee all agents' position states to remain in a bounded region. The algorithm gains need not to be predesigned and can be selected by each agent using its own and neighbours' information. By a model transformation, the original nonlinear time-varying system is converted into a linear time-varying one with a nonlinear error term. Based on the properties of stochastic matrices, it is shown that the optimization problem can be solved as long as the communication topologies are jointly strongly connected and balanced. Numerical examples are given to show the obtained theoretical results.

Equations

$$\displaystyle\begin{array}[]{lll}{r}_{i}((k+1)T)&=&r_{i}(kT)+v_{i}(kT)T\\ {v}_{i}((k+1)T)&=&u_{i}(kT)\end{array}$$

$$\displaystyle\mathrm{S}_{V_{i}}(x)=\left\{\begin{array}[]{lll}\frac{\displaystyle x}{\displaystyle\|x\|}\max\limits_{{0\leq\beta\leq\|x\|}}{\Big{\{}}\beta{\big{|}}\frac{\displaystyle\alpha\beta x}{\displaystyle\|x\|}\in V_{i},\forall 0\leq\alpha\leq 1{\Big{\}}},\\ \mbox{if~{}~{}}0\neq x\in\mathbb{R}^{m},\\ 0,\mbox{otherwise}.\end{array}\right.$$

$$\displaystyle\begin{array}[]{lll}\mathrm{minimize}~{}~{}\sum_{i=1}^{n}f_{i}(s)\\ \mathrm{subject~{}to}~{}~{}s\in\mathbb{R}^{m},\end{array}$$

$$\displaystyle\begin{array}[]{lll}{y}_{i}(k+1)={y}_{i}(k)+\mathrm{arctan}(e^{\|r_{i}(k)\|})T,y_{i}(0)>0,\\ \pi_{i}(k)=\sum\limits_{j\in\mathcal{N}_{i}(k)}a_{ij}(k)(r_{j}(k)-r_{i}(k))T,\\ q_{i}(k)=v_{i}(k)-p_{i}(k)v_{i}(k)T+\frac{p_{i}(k)}{2}\pi_{i}(k),\\ w_{i}(k)=r_{i}(k)+\frac{2}{p_{i}(k)}{v}_{i}(k)-v_{i}(k)T+\pi_{i}(k),\\ \theta_{i}(k)=\left\{\begin{array}[]{lll}0,\mbox{~{}if~{}}\sqrt{y_{i}(k)}<\|\nabla f_{i}(w_{i}(k))\|^{2},\\ 0,\mbox{~{}if~{}}q_{i}(k)-\frac{p_{i}(k)\nabla f_{i}(w_{i}(k))}{2\sqrt{y_{i}(k)}}\\ \neq\mathrm{S}_{V_{i}}[q_{i}(k)-\frac{p_{i}(k)\nabla f_{i}(w_{i}(k))}{2\sqrt{y_{i}(k)}}],\\ \frac{p_{i}(k)\nabla f_{i}(w_{i}(k))}{2\sqrt{y_{i}(k)}},\mbox{~{}otherwise,}\end{array}\right.\\ u_{i}(k)=\mathrm{S}_{V_{i}}[q_{i}(k)-\theta_{i}(k)],\end{array}$$

$$\displaystyle\begin{array}[]{lll}u_{i}(k)=\mathrm{S}_{V_{i}}[q_{i}(k)-\theta_{i}(k)]=\sigma_{i}(k)[q_{i}(k)-\theta_{i}(k)].\end{array}$$

$$\displaystyle\begin{array}[]{lll}u_{i}(k)=\sigma_{i}(k)[v_{i}(k)-p_{i}(k)v_{i}(k)T+\frac{p_{i}(k)}{2}\pi_{i}(k)-\theta_{i}(k)]\\ =v_{i}(k)-b_{i}(k)v_{i}(k)T+\frac{p_{i}(k)\sigma_{i}(k)}{2}\pi_{i}(k)-\sigma_{i}(k)\theta_{i}(k).\end{array}$$

$$\displaystyle\begin{array}[]{lll}r_{i}(k+1)=r_{i}(k)+\frac{p_{i}(k)}{2}[z_{i}(k)-r_{i}(k)]T\end{array}$$

$$\displaystyle\begin{array}[]{lll}z_{i}(k+1)=z_{i}(k)+[\frac{p_{i}(k)T}{2}-1+\frac{p_{i}(k)}{p_{i}(k+1)}-\frac{p_{i}(k)b_{i}(k)T}{p_{i}(k+1)}]\\ \times[z_{i}(k)-r_{i}(k)]+\frac{p_{i}(k)\sigma_{i}(k)}{p_{i}(k+1)}\pi_{i}(k)-\frac{2\sigma_{i}(k)}{p_{i}(k+1)}\theta_{i}(k).\end{array}$$

$$\displaystyle\begin{array}[]{lll}w_{i}(k)=z_{i}(k)-\frac{p_{i}(k)}{2}[z_{i}(k)-r_{i}(k)]T+\pi_{i}(k).\end{array}$$

$$\displaystyle\begin{array}[]{lll}z_{i}(k+1)=w_{i}(k)-\frac{\nabla f_{i}(w_{i}(k))}{\sqrt{y_{i}(k)}}.\end{array}$$

$$\displaystyle\begin{array}[]{lll}&\phi(k+1)=\Psi(k)\phi(k)-\nabla F(k),\end{array}$$

$$E_i(k+1)E_i^{-1}(k)=\left[\begin{array}{cc}1& 0\\ 1-\frac{p_i(k)}{p_i(k+1)}& \frac{p_i(k)}{p_i(k+1)}\end{array}\right].$$

$$b_i(k+1)\ge p_i(k+1)\ge b_i(k)\ge p_i(k)>0$$

$$\displaystyle\begin{array}[]{lll}\|z_{i}(k+1)-s\|^{2}\\ =\|w_{i}(k)-s\|^{2}+{\big{\|}}\frac{\nabla f_{i}(w_{i}(k))}{\sqrt{y_{i}(k)}}{\big{\|}}^{2}-2(w_{i}(k)-s)^{T}\frac{\nabla f_{i}(w_{i}(k))}{\sqrt{y_{i}(k)}}\\ \leq\|w_{i}(k)-s\|^{2}+\frac{1}{\sqrt{y_{i}(k)}}-2(w_{i}-s)^{T}\frac{\nabla f_{i}(w_{i}(k))}{\sqrt{y_{i}(k)}}.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\|z_{i}(k+1)-s\|^{2}\leq\|w_{i}(k)-s\|^{2}+\frac{1}{\sqrt{y_{i}(k)}}\\ +2[f_{i}(s)-f_{i}(s_{i})+f_{i}(s_{i})-f_{i}(w_{i}(k))]/\sqrt{y_{i}(k)}\end{array}$$

$$\displaystyle\begin{array}[]{lll}\|z_{i}(k+1)-s\|^{2}\leq\|w_{i}(k)-s\|^{2}+\frac{2}{\sqrt{kT}}\\ +4[f_{i}(s)-f_{i}(s_{i})]/\sqrt{kT}+[f_{i}(s_{i})-f_{i}(w_{i}(k))]/\sqrt{kT}\end{array}$$

$$\displaystyle\begin{array}[]{lll}\phi(k_{m+1})=\Gamma(k_{m+1}-1,k_{m})\phi(k_{m})\\ -\sum_{j=k_{m}+1}^{k_{m+1}-1}\Gamma(k_{m+1}-1,j)\nabla F(j-1)-\nabla F(k_{m+1}-1).\end{array}$$

$$\displaystyle\begin{array}[]{lll}\|\sum_{j=k_{m}+1}^{k_{m+1}-1}\Gamma(k_{m+1}-1,j)\nabla F(j-1)\\ -\nabla F(k_{m+1}-1)\|\leq(\eta+1)\epsilon\end{array}$$

$$\displaystyle\begin{array}[]{lll}\|\phi_{i}(k_{m+4n})-\sum_{j\neq i_{0}}\lfloor\Gamma(k_{m+4n}-1,k_{m})\rfloor_{ij}\phi_{j}(k_{m})\\ +\lfloor\Gamma(k_{m+4n}-1,k_{m})\rfloor_{ii_{0}}\phi_{i_{0}}(k_{m})\|\leq{4n(\eta+1)\epsilon}.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\max_{i}\phi_{i}(k_{m+4n})\leq(1-\hat{\mu})\max_{i}\phi_{i}(k_{m})+\hat{\mu}\phi_{i_{0}}(k_{m})+4n(\eta+1)\epsilon,\\ \min_{i}\phi_{i}(k_{m+4n})\geq(1-\hat{\mu})\min_{i}\phi_{i}(k_{m})+\hat{\mu}\phi_{i_{0}}(k_{m})-4n(\eta+1)\epsilon,\end{array}$$

$$\displaystyle\begin{array}[]{lll}\max_{i}\phi_{i}(k_{m+4n})-\min_{i}\phi_{i}(k_{m+4n})\\ \leq(1-\hat{\mu})[\max_{i}\phi_{i}(k_{m})-\min_{i}\phi_{i}(k_{m})]+8n(\eta+1)\epsilon.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\max_{i}\phi_{i}(k_{m+4n})-\min_{i}\phi_{i}(k_{m+4n})\\ \leq[(1-\hat{\mu})/\hat{\mu}+1]8n(\eta+1)\epsilon\leq 8n(\eta+1)\epsilon/\hat{\mu}\leq 16n(\eta+1)\epsilon/\hat{\mu}.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\max_{i}\phi_{i}(k_{m+4n})-\min_{i}\phi_{i}(k_{m+4n})\\ -[\max_{i}\phi_{i}(k_{m})-\min_{i}\phi_{i}(k_{m})]<0.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\max_{i}\phi_{i}(k_{m+4n})-\min_{i}\phi_{i}(k_{m+4n})\\ -[\max_{i}\phi_{i}(k_{m})-\min_{i}\phi_{i}(k_{m})]<-8n(\eta+1)\epsilon.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\max_{i}\phi_{i}(k_{m+4n})-\min_{i}\phi_{i}(k_{m+4n})\leq 16n(\eta+1)\epsilon/\hat{\mu}.\end{array}$$

$$\displaystyle\begin{array}[]{lll}\phi^{*}(k+1)=\phi^{*}(k)-\frac{1}{2n}\mathbf{1}^{T}\nabla F(k).\end{array}$$

$$\displaystyle\begin{array}[]{lll}&&V_{1}(k+1)\\ &=&\|\phi^{*}(k+1)-P_{X}(\phi^{*}(k))+P_{X}(\phi^{*}(k))-P_{X}(\phi^{*}(k+1))\|^{2}\\ &=&\|\phi^{*}(k+1)-P_{X}(\phi^{*}(k))\|^{2}\\ &+&\|P_{X}(\phi^{*}(k))-P_{X}(\phi^{*}(k+1))\|^{2}\\ &+&2[\phi^{*}(k+1)-P_{X}(\phi^{*}(k))]^{T}[P_{X}(\phi^{*}(k))-P_{X}(\phi^{*}(k+1))]\\ &\leq&\|\phi^{*}(k+1)-P_{X}(\phi^{*}(k))\|^{2}+\|\phi^{*}(k)-\phi^{*}(k+1)\|^{2}\\ &\leq&\|\phi^{*}(k+1)-P_{X}(\phi^{*}(k))\|^{2}+\|\frac{1}{2n}\mathbf{1}^{T}\nabla F(k)\|^{2}.\end{array}$$

$$\displaystyle\begin{array}[]{lll}&&V_{1}(k+1)\\ &\leq&\|\phi^{*}(k)-\frac{1}{2n}\mathbf{1}^{T}\nabla F(k)-P_{X}(\phi^{*}(k))\|^{2}+\|\frac{1}{2n}\mathbf{1}^{T}\nabla F(k)\|^{2}\\ &\leq&\|\phi^{*}(k)-P_{X}(\phi^{*}(k))\|^{2}-\frac{1}{n}(\phi^{*}(k)-P_{X}(\phi^{*}(k)))^{T}\\ &\times&\sum_{i=1}^{n}\frac{\nabla f_{i}(w_{i}(k))^{T}}{\sqrt{y_{i}(k)}}+\|\frac{1}{n}\mathbf{1}^{T}\nabla F(k)\|^{2}\\ &\leq&\|\phi^{*}(k)-P_{X}(\phi^{*}(k))\|^{2}-\frac{1}{n}\sum_{i=1}^{n}(\phi^{*}(k)-P_{X}(\phi^{*}(k)))^{T}\\ &\times&\frac{\nabla f_{i}(w_{i}(k))^{T}}{\sqrt{y_{i}(k)}}+\frac{\gamma}{k}\end{array}$$

$$\displaystyle\begin{array}[]{lll}&&V_{1}(k+1)-V_{1}(k)\\ &\leq&-\frac{1}{n}\sum_{i=1}^{n}(\phi^{*}(k)-w_{i}(k)+w_{i}(k)-P_{X}(\phi^{*}(k)))^{T}\\ &\times&\frac{\nabla f_{i}(w_{i}(k))^{T}}{\sqrt{y_{i}(k)}}+\frac{\gamma}{k}\\ &\leq&-\frac{1}{n}\sum_{i=1}^{n}(w_{i}(k)-P_{X}(\phi^{*}(k)))^{T}\frac{\nabla f_{i}(w_{i}(k))^{T}}{\sqrt{y_{i}(k)}}\\ &+&\frac{\gamma}{k}+\frac{\tilde{\gamma}\epsilon_{1}}{\sqrt{k}}\\ &\leq&-\frac{1}{n}\sum_{i=1}^{n}\frac{1}{\sqrt{y_{i}(k)}}[f_{i}(w_{i}(k))-f_{i}(P_{X}(\phi^{*}(k)))]\\ &+&\frac{\gamma}{k}+\frac{\tilde{\gamma}\epsilon_{1}}{\sqrt{k}}\\ &\leq&-\frac{1}{n}\sum_{i=1}^{n}\frac{1}{\sqrt{y_{i}(k)}}[f_{i}(w_{i}(k))-f_{i}(\phi^{*}(k))\\ &+&f_{i}(\phi^{*}(k))-f_{i}(P_{X}(\phi^{*}(k)))]+\frac{\gamma}{k}+\frac{\tilde{\gamma}\epsilon_{1}}{\sqrt{k}}\\ &\leq&-\frac{1}{n}\sum_{i=1}^{n}\frac{1}{\sqrt{y_{i}(k)}}[f_{i}(\phi^{*}(k))-f_{i}(P_{X}(\phi^{*}(k)))]\\ &+&\frac{\gamma}{k}+\frac{2\tilde{\gamma}\epsilon_{1}}{\sqrt{k}}\end{array}$$

$$\displaystyle\begin{array}[]{lll}&&V_{1}(k+1)-V_{1}(k)\\ &\leq&-\frac{1}{n}\sum_{i=1}^{n}\frac{1}{\sqrt{y_{1}(k)}}[f_{i}(\phi^{*}(k))-f_{i}(P_{X}(\phi^{*}(k)))]\\ &-&\frac{1}{n}\sum_{i=1}^{n}\frac{1}{\sqrt{y_{1}(k)}}(\frac{\sqrt{y_{1}(k)}}{\sqrt{y_{i}(k)}}-1)[f_{i}(\phi^{*}(k))\\ &-&f_{i}(P_{X}(\phi^{*}(k)))]+\frac{\gamma\epsilon_{1}}{\sqrt{k}}+\frac{2\tilde{\gamma}\epsilon_{1}}{\sqrt{k}}\\ &\leq&-\frac{1}{n\sqrt{y_{1}(k)}}[\sum_{i=1}^{n}(f_{i}(\phi^{*}(k))-f_{i}(P_{X}(\phi^{*}(k))))\\ &-&n\epsilon_{1}]+\frac{\gamma\epsilon_{1}}{\sqrt{k}}+\frac{2\tilde{\gamma}\epsilon_{1}}{\sqrt{k}}\\ &\leq&-\frac{1}{2n\sqrt{kT}}[\sum_{i=1}^{n}(f_{i}(\phi^{*}(k))-f_{i}(P_{X}(\phi^{*}(k))))-\hat{\gamma}\epsilon_{1}]\end{array}$$

Full text

Keywords: Distributed Optimization, Nonconvex Constraint Sets, Nonuniform Position Constraint Sets, Nonuniform Step-Sizes

I Introduction

As an important branch of distributed control theory, the distributed optimization problem of multi-agent systems has attracted more and more attention from the control community [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]. For example, articles [1, 5, 6] studied the distributed optimization problems with and without convex constraints by a projection algorithm and showed that all agents reach a consensus while optimizing the given team performance functions when the communication topologies are jointly strongly connected and balanced. By introducing an integrator term in the algorithm for each agent, articles [2, 3, 4] solved the distributed optimization problem without using the vanishing stepsizes when the communication topologies are strongly connected balanced directed graphs.

When the constraints are taken into account, most of the existing works assumed the constraint sets to be convex and few works have paid attention to the case of nonconvex constraints. In practical applications, the constraint sets might not be convex, e.g., the velocities of the quadrotors. It is meaningful to study the distributed optimization problem with nonconvex constraint sets. Moreover, most of the existing works assumed the stepsizes of the gradients or subgradients to be uniform at any instant, which made the algorithms there not fully distributed. Article [14] took nonuniform stepsizes into account for the distributed optimization problem in a stochastic setting, but through computing its mathematical expectation, the proposed algorithms are essentially of the uniform-stepsize ones. Articles [16, 17] introduced a kind of state-dependent stepsizes to enable each agent to be able to use its own and neighors’ information to optimize the team performance function without using predesigned stepsizes, but the constraint sets were assumed to be convex and nonconvex constraint sets were not considered.

In this paper, we focus on distributed optimization of multi-agent systems with nonconvex velocity constraints, nonuniform position constraints and nonuniform stepsizes. In [18], the consensus problem of multi-agent system with nonconvex velocity constraints was studied, but due to the nonlinearity of the optimization term and the unbalance of the agent interaction there caused by the nonlinear constraint operators, the results cannot be applied to solve the optimization problem. Besides, most of the existing works only considered the position constraints and none has taken into account the position and velocity constraints simultaneously. Due to the nonlinearity caused by the nonconvex velocity constraints, the nature of the system is totally changed, which makes the existing approaches no longer valid for our setting. To solve the nonconvex velocity-constrained optimization problem, we first propose a distributed constrained algorithm by introducing a switching mechanism to guarantee all agents’ position states to remain in a bounded region. Thereinto, the algorithm gains need not to be predesigned and can be selected by each agent using its own and neighbours’ information. Second, by using a model transformation, we convert the original nonlinear time-varying system into a linear time-varying one with a nonlinear error term. Third, based on the properties of stochastic matrices, it is shown that the effects of the error term on the consensus convergence of the system vanish to zero as time evolves and the optimization problem can be solved if the communication topologies are jointly strongly connected and balanced. After that, we extend the results to the case when there further exist nonuniform convex position constraints. A distributed constrained algorithm using the projection operator in combination with the switching mechanism is proposed and it is shown that the optimization problem can be solved by combing the above analysis approach with that in [15]. Compared with the existing works, the main contribution of this note is that the three challenges, namely, nonconvex velocity constraints, nonuniform position constraints and nonuniform stepsizes, are addressed simultaneously for distributed optimization of multi-agent systems. In fact, each of the above challenges is rarely addressed in the literature, not to mention a combination of them. Also the agent dynamics under consideration are in the form of double integrators instead of single integrators. In addition, the proposed algorithms are fully distributed and can be implemented by using only local information and local interaction.

II Preliminaries

In this section, some preliminary results about graph theory, projection operator and stochastic matrices are introduced (see [19], [21] and [22]). Let $\mathcal{G}(\mathcal{I},\mathcal{E})$ be a directed graph, where $\mathcal{I}=\{1,\cdots,n\}$ is the node set, and $\mathcal{E}\subseteq\mathcal{I}\times\mathcal{I}$ is the edge set. An edge of $\mathcal{G}$, denoted by $(j,i)$, represents the information flow from node $j$ to node $i$. It is assumed that $(i,i)\notin\mathcal{E}$ for all $i$. The neighbor set of node $i$ is denoted by $\mathcal{N}_{i}=\{j\in\mathcal{I}:(j,i)\in\mathcal{E}\}$. The edge weight of $(j,i)$ is defined such that $a_{ij}>\mu_{c}$ for some constant $\mu_{c}>0$ if $(j,i)\in\mathcal{E}$ and $a_{ij}=0$ otherwise. The Laplacian of the directed graph $\mathcal{G}$, denoted by $L$, is defined as $\lfloor{L}\rfloor_{ii}=\sum_{j=1}^{n}a_{ij}$ and $\lfloor{L}\rfloor_{ij}=-a_{ij}$ for all $i\neq j$, where $\lfloor{L}\rfloor_{ii}$ and $\lfloor{L}\rfloor_{ij}$ denote the $ii$th and $ij$th entries of the matrix ${L}$. For a given group of nodes, the union of a set of graphs is a graph whose edge sets are the union of the edge sets of the graphs in the set. A directed path is a sequence of ordered edges of the form $({i_{1}},{i_{2}}),({i_{2}},{i_{3}}),\cdots,$ where ${i_{j}}\in\mathcal{I}$. A directed graph is strongly connected if there is a directed path from every node to every other node.

Lemma 1**.**

[21] Suppose that $Y\neq\emptyset$ is a closed convex set in $\mathbb{R}^{m}$. The following statements hold.

(1) For any $y\in\mathbb{R}^{m}$, $\|y-P_{Y}(y)\|$ is continuous with respect to $y$ and $\nabla\frac{1}{2}\|y-P_{Y}(y)\|^{2}=y-P_{Y}(y)$ where $P_{Y}$ denotes the projection operator defined as $P_{Y}(y)=\mathrm{arg}\min\limits_{{s}\in Y}\|y-{s}\|$;

(2) For any $y,z\in\mathbb{R}^{m}$ and all $\xi\in Y$, $[y-P_{Y}(y)]^{T}(y-\xi)\geq 0$, $\|P_{Y}(y)-\xi\|^{2}\leq\|y-\xi\|^{2}-\|P_{Y}(y)-y\|^{2}$ and $\|P_{Y}(y)-P_{Y}(z)\|\leq\|y-z\|.$

Given $C\in\mathbb{R}^{n\times r}$, $C$ is nonnegative ($C\geq 0$) if all its elements are nonnegative, and $C$ is positive ($C>0$) if all its elements $c_{ij}$ are positive. If a nonnegative matrix $C\in\mathbb{R}^{n\times n}$ satisfies $C\textbf{1}=\textbf{1}$, then it is stochastic.

III Model

Consider a multi-agent system with $n$ agents. Let $\mathcal{G}(kT)$ denote its communication graph, where $k$ is the discrete time index and $T$ is the sampling period, ${L}(kT)$ denote the Laplacian of $\mathcal{G}(kT)$ and $\mathcal{N}_{i}(kT)$ denote the neighbor set whose information agent $i$ has access to. Suppose that each agent has the following dynamics

[TABLE]

where $r_{i}(kT)\in\mathbb{R}^{m}$, $v_{i}(kT)\in\mathbb{R}^{m}$ and $u_{i}(kT)\in\mathbb{R}^{m}$ are the position, velocity and the control input of agent $i$ for some positive integer $m$. In the following, all “$(kT)$” will be replaced by “$(k)$” when no confusion arises. In reality, the agent velocities are often constrained to remain in nonconvex sets. For example, quadrotors can move towards every direction but the maximum velocities in different directions might be different and all of their possible velocities do not necessarily form a convex set. To this end, we assume that $v_{i}(k)\in V_{i}\subseteq\mathbb{R}^{m}$ where each $V_{i}$ is a nonconvex set that is known to only agent $i$. Before giving the specific assumption, we need first introduce a constraint operator that will also be used in our algorithms.

Define

[TABLE]

The operator $\mathrm{S}_{V_{i}}(x)$ was proposed in [18]. Its role is to find the vector with the largest magnitude such that $\mathrm{S}_{V_{i}}(x)$ has the same direction as $x$, $\|\mathrm{S}_{V_{i}}(x)\|\leq\|x\|$ and $\alpha\mathrm{S}_{V_{i}}(x)\in V_{i}$ for all $0\leq\alpha\leq 1$.

Assumption 1**.**

[18] Let $V_{i}\subseteq\mathbb{R}^{m},i=1,\cdots,n$, be nonempty bounded closed sets such that $0\in V_{i}$, $\max_{x\in V_{i}}\|\mathrm{S}_{V_{i}}(x)\|=\bar{\rho}_{i}>0$ and $\inf_{x\notin V_{i}}\|\mathrm{S}_{V_{i}}(x)\|=\underline{\rho}_{i}>0$ for all $i$, where $\bar{\rho}_{i}$ and $\underline{\rho}_{i}$ are two positive constants, and $\inf_{x\notin V_{i}}\|\mathrm{S}_{V_{i}}(x)\|$ denotes the infimum of $\|\mathrm{S}_{V_{i}}(x)\|$ when $x\notin V_{i}$.

In Assumption 1, we do not require each $V_{i}$ to be convex. What we require on $V_{i}$ is that $V_{i}$ is bounded and the distance from any point outside $V_{i}$ to the origin is lower bounded by a positive constant.

IV Problem Formulation and Algorithm

Our objective is to design a distributed algorithm to make all agents cooperatively find the optimal state of the optimization problem

[TABLE]

where $f_{i}(s):\mathbb{R}^{m}\rightarrow\mathbb{R}$ denotes the differentiable convex local objective function that is known to only agent $i$.

Assumption 2**.**

[16] Each set X_{i}\triangleq{\Big{\{}}x{\Big{|}}\nabla f_{i}(x)=0{\Big{\}}}, $i\in\mathcal{I}$, is nonempty and bounded.

Lemma 2**.**

[20] Let $f_{0}(\chi):\mathbb{R}^{m}\rightarrow\mathbb{R}$ be a differentiable convex function. $f_{0}(\chi)$ is minimized if and only if $\nabla f_{0}(\chi)=0$.

From Lemma 2, each $X_{i}$ is the optimal set of the local objective function $f_{i}(x)$. Let $X$ be the optimal set of $\sum_{i=1}^{n}f_{i}(x)$. We have the following lemma.

Lemma 3**.**

[16] Under Assumptions 1 and 2, all $X_{i}$, $i\in\mathcal{I}$, and $X$ are nonempty closed bounded convex sets.

Lemma 4**.**

[16] Let $f(s):\Xi\mapsto\mathbb{R}$ be a differentiable convex function and $Y$ be its minimum set in $\Xi$, where $\Xi\subseteq\mathbb{R}^{m}$ is a closed convex set. Suppose that $Y\subseteq\Xi$ is closed and bounded. For any $z=\lambda P_{Y}(y)+(1-\lambda)y$ with $\lambda\in(0,1)$, $0<\nabla f(z)^{T}\frac{y-P_{Y}(y)}{\|y-P_{Y}(y)\|}\leq\nabla f(y)^{T}\frac{y-P_{Y}(y)}{\|y-P_{Y}(y)\|}$ for any $y\in\Xi-Y$.

Lemma 5**.**

[16] Under Assumption 2, $\lim_{\|y\|\rightarrow+\infty}f_{i}(y)=+\infty$ for all $i$ and accordingly $\lim_{\|y\|\rightarrow+\infty}\sum_{i=1}^{n}f_{i}(y)=+\infty$.

To solve the optimization problem (7) in a distributed manner, we give the following algorithm for each agent by

[TABLE]

for all $k\geq 0$, where $v_{i}(0)\in{V_{i}}$ and $p_{i}(k)>0$ is the feedback damping gain.

In (18), $q_{i}(k)$ is a linear combination of the agent states, which is used to make all agents converge to a consensus point, and $\theta_{i}(k)$ contains a switching mechanism, where the first switching rule is used to guarantee all agents’ position states to remain in a bounded region while the second switching rule is used to guarantee the balance of the optimal convergence rates of all agents. Specifically, in $\theta_{i}(k)$, the stepsize of the gradient, $\frac{1}{\sqrt{y_{i}(k)}}$, which is constructed based on only the position states, has two features: one is $\lim_{k\rightarrow+\infty}\frac{1}{\sqrt{y_{i}(k)}}=0$ and the other is $\lim_{k\rightarrow+\infty}\frac{\sqrt{y_{j}(k)}}{\sqrt{y_{i}(k)}}=1$ for all $i,j$, which will be shown later. The role of the stepsize $\frac{1}{\sqrt{y_{i}(k)}}$ is to minimize the effects of the gradient on the consensus convergence and keep the balance of the optimal convergence rates of all agents. $w_{i}(k)$ is a linear combination of the states and it is a key variable to determine the consensus behavior of the system dynamics, which will also be shown later.

In this note, our analysis is for the general $m$ case. When no confusion arises, the equations are given in the form of $m=1$ for simplicity of derivation expression.

V Stability analysis

Due to the nonconvexity of the operator $\mathrm{S}_{V_{i}}$ and the coexisting nonlinearities of the operator $\mathrm{S}_{V_{i}}$ and the gradient $\nabla f_{i}$, the system (3) with (18) is distinctly different from the distributed optimization problems with convex constraints and the consensus problem with nonconvex constraints studied in [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]. The approaches in [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18] cannot be directly applied. To study the system (3) with (18), we first make some model transformations. Let $\sigma_{i}(k)=\mathrm{S}_{V_{i}}[q_{i}(k)-\theta_{i}(k)]/[q_{i}(k)-\theta_{i}(k)].$ When $q_{i}(k)-\theta_{i}(k)=0$, $\mathrm{S}_{V_{i}}[q_{i}(k)-\theta_{i}(k)]=q_{i}(k)-\theta_{i}(k)$ and hence define $\sigma_{i}(k)=1$. Then $u_{i}(k)$ can be transformed into the form:

[TABLE]

As $\sigma_{i}(k)$ is a time-varying scaling factor, it is hard to perform analysis directly on the double integrator system with the control input in such a form. To proceed, we define two new variables $b_{i}(k)\in\mathbb{R}$ and $z_{i}(k)\in\mathbb{R}^{m}$ satisfying $1-b_{i}(k)T=\sigma_{i}(k)(1-p_{i}(k)T)$ and $z_{i}(k)=r_{i}(k)+\frac{2}{p_{i}(k)}v_{i}(k)$. Note that

[TABLE]

Rewriting the system (3) with (18), we have

[TABLE]

and

[TABLE]

It can easily be observed that the sums of the coefficients of $r_{i}(k)$ and $z_{i}(k)$ in (22) and (25) are both equal to $1$, which will be used for the system analysis. It should be noted here that when $\theta_{i}(k)\neq 0$, it follows from (18) that $q_{i}(k)-\frac{p_{i}(k)\nabla f_{i}(w_{i}(k))}{2\sqrt{y_{i}(k)}}=\mathrm{S}_{V_{i}}[q_{i}(k)-\frac{p_{i}(k)\nabla f_{i}(w_{i}(k))}{2\sqrt{y_{i}(k)}}]$ and hence $\sigma_{i}(k)=1$, i.e., $b_{i}(k)=p_{i}(k)$. When $\theta_{i}(k)\neq 0$, if $p_{i}(k+1)=b_{i}(k)$, we have that

[TABLE]

Rewriting (25) using $w_{i}(k)$ when $\theta_{i}(k)\neq 0$ and $p_{i}(k+1)=b_{i}(k)$, we have that

[TABLE]

Let $\phi(k)=[r_{1}(k)^{T},z_{1}(k)^{T},\cdots,r_{n}(k)^{T},z_{n}(k)^{T}]^{T}$, $E_{i}(k)=\begin{bmatrix}1&0\\ 1&\frac{2}{p_{i}(k)}\end{bmatrix}$, $A_{i}(k)=\begin{bmatrix}1-\frac{p_{i}(k)T}{2}&\frac{p_{i}(k)T}{2}\\ {b_{i}(k)T}-\frac{p_{i}(k)T}{2}&1-{b_{i}(k)T}+\frac{p_{i}(k)T}{2}\end{bmatrix}$, $E(k)=\mathrm{diag}\{E_{1}(k),\cdots,E_{n}(k)\}$ and $A(k)=\mathrm{diag}\{A_{1}(k),\cdots,A_{n}(k)\}$ where $E(k)$ and $A(k)$ are block diagonal matrices with their diagonal blocks equal to the matrices $E_{i}(k)$ and $A_{i}(k)$ respectively. Let $\Lambda(k)=\mathrm{diag}\{\sigma_{1}(k),\cdots,\sigma_{n}(k)\}$, $W=\begin{bmatrix}0&0\\ T&0\end{bmatrix}$ and $\nabla F(k)=[0^{T},\frac{2\sigma_{1}(k)}{p_{1}(k+1)}\theta_{1}(k)^{T},\cdots,0^{T},\frac{2\sigma_{n}(k)}{p_{n}(k+1)}\theta_{n}(k)^{T}]^{T}$.

It follows that

[TABLE]

where $\Psi(k)=E(k+1)E(k)^{-1}[A(k)-\Lambda(k)L(k)\otimes W]$ and ${}^{\prime}\otimes^{\prime}$ denotes the Kronecker product.

Remark 1**.**

From the definition of $\sigma_{i}(k)$, each $\sigma_{i}(k)$ is time-varying and might not be uniform for all $i$ due to the nonconvex constraints, and hence the gradient weights might be nonuniform as well. From the view point of intuition, this might make the algorithm fail to solve the team optimization problem. Most of the existing works assumed the gradient weight to be uniform. Though articles [16, 17] have considered the case of nonuniform stepsizes, it is still unclear how to deal with the case in the system (31) when the nonuniform stepsizes and the time-varying scaling factors are taken into account simultaneously.

In the following, we will first study the properties of the system matrices of (31) in Lemmas 6-8, and then the consensus and optimal convergence of (31) in Lemmas 9-10 and Theorem 1. Specifically, Lemma 6 shows that the system matrices and the transition matrices are both stochastic. Lemma 7 shows that all agents’ position states remain in a bounded region and each nonzero entry of these matrices is lower bounded by a positive constant by exploiting the swiching mechanism in the algorithm and the conditions given in the assumptions. Lemma 8 shows that there exist at least a column of the transition matrices each entry of which is lower bounded by a positive constant. Based on Lemma 8, Lemma 9 shows that all agents reach a consensus as time evolves. Lemma 10 extends the continuous-time results of [17] on gradient gains to the discrete-time system and shows that the ratio of all stepsizes finally tends to $1$ as time evolves. Based on Lemmas 9 and 10, Theorem 1 shows that the team optimization function is minimized as time evolves.

Assumption 3**.**

Suppose that $\frac{1}{T}>p_{i}(k+1)=b_{i}(k)>0$ for all $k\geq 0$ and all $i$, and there exist a constant $d_{i}>0$ such that $p_{i}(k)>2d_{i}>2\lfloor{L}({k})\rfloor_{ii}$ for all $i$ and all $k\geq 0$.

Assumption 3 actually gives a design rule for the algorithm, under which it will be shown that the transition matrices are all stochastic. Also, the constants $d_{i}$ always exist if $2\sum_{j\in\mathcal{N}_{i}(k)}a_{ij}(k)<p_{i}(0)$, which can be concluded from the proof of Lemma 6.

To implement the algorithm (18) under Assumption 3, we need to know the quantities, $p_{i}(k)$, $w_{i}(k)$, $q_{i}(k)$, and $\theta_{i}(k)$. Since $p_{i}(k)=b_{i}(k-1)$, $p_{i}(k)$ can be obtained through computing $b_{i}(k-1)=[1-\sigma_{i}(k-1)(1-p_{i}(k-1)T)]/T$, where $\sigma_{i}(k-1)$ and $p_{i}(k-1)$ are both known at time instant $k$. In particular, for $k=0$, $p_{i}(0)$ can be adopted properly satisfying Assumption 3. Note that $w_{i}(k)$ and $q_{i}(k)$ are actually linear combinations of the variables $v_{i}(k)$, $r_{i}(k)$ and $r_{j}(k)$ for $j\in\mathcal{N}_{i}(k)$. Based on the obtained $p_{i}(k)$, $w_{i}(k)$ and $q_{i}(k)$ can be easily computed. As the variable $\theta_{i}(k)$ is dependent on the switching mechanism of the algorithm, we need judge the switching rules by computing $\sqrt{y_{i}(k)}$, $\|\nabla f_{i}(w_{i}(k))\|^{2}$, $q_{i}(k)-\frac{p_{i}(k)\nabla f_{i}(w_{i}(k))}{2\sqrt{y_{i}(k)}}$, and $\mathrm{S}_{V_{i}}[q_{i}(k)-\frac{p_{i}(k)\nabla f_{i}(w_{i}(k))}{2\sqrt{y_{i}(k)}}]$ where $y_{i}(k)$ is known at time instant $k$ so as to obtain the variable $\theta_{i}(k)$. Though the algorithm computation looks a bit complex due to the existence of the switching mechanism, the algorithm does not require intermediate variables to be transmitted and it is a fully distributed algorithm.

Let $\Gamma(k,s)=\prod_{i=s}^{k}\Psi(i)=\Psi(k)\cdots\Psi(s)$ be the transition matrix of the system (31).

Lemma 6**.**

Under Assumptions 1, 2 and 3, $\Psi(k)$ and $\Gamma(k,s)$ are stochastic matrices for any $k\geq s\geq 0$.

Proof: By simple calculations,

[TABLE]

From the definition of $b_{i}(k)$, when $p_{i}(k)>0$, it is easy to see that $1>b_{i}(k)\geq p_{i}(k)$ and hence

[TABLE]

for all $k$ under Assumption 3. Thus $E(k+1)E(k)^{-1}$ is a stochastic matrix. Note that $L(k)\mathbf{1}=0$ from the definition of the graph Laplacian. It is easy to see that $[A(k)-\Lambda(k)L(k)\otimes W]\mathbf{1}=\mathbf{1}$. Note that $0\leq\sigma_{i}(k)\leq 1$. Under Assumption 3, $b_{i}(k)T-\frac{p_{i}(k)T}{2}-\frac{p_{i}(k)\sigma_{i}(k)}{b_{i}(k)}d_{i}T\geq\frac{p_{i}(k)T}{2}-d_{i}T>0$ for all $i,k$. It can be easily checked that each entry of $A(k)-\Lambda(k)L(k)\otimes W$ is nonnegative and hence $A(k)-\Lambda(k)L(k)\otimes W$ is a stochastic matrix. Therefore, $\Psi(k)$ and $\Gamma(k,s)$ are both stochastic matrices for any $k\geq s\geq 0$.

Assumption 4**.**

Suppose that there exist an infinite time sequence of $k_{0},k_{1},k_{2},\cdots$ and a positive integer $\eta$ such that $k_{0}=0$, $0<k_{m+1}-k_{m}\leq\eta$ for all $m$ and the union of the graphs $\mathcal{G}(k_{m}),\mathcal{G}(k_{m}+1),\cdots,\mathcal{G}(k_{m+1}-1)$ is strongly connected.

Assumption 4 ensures that the agents keep communication with each other persistently, which is a necessary condition for all agents to reach a consensus and minimize the team objective function.

Lemma 7**.**

Under Assumptions 1, 2, 3 and 4,

• (1)

$\|r_{i}(k)\|<\rho$ and $\|z_{i}(k)\|<\rho$ for all $i,k$ and some constant $\rho>0$;

• (2)

each $\sigma_{i}(k)$ and each $\frac{p_{i}(k)}{p_{i}(k+1)}$ are both lower bounded by a positive constant for all $i,k$;

• (3)

each nonzero entry of $A(k)-\Lambda(k)L(k)\otimes W$ is lower bounded by a positive constant.

Proof: Construct the Lyapunov function candidate $V(k)=\max_{i}\{\|r_{i}(k)-s\|,\|z_{i}(k)-s\|\}$ for some $s\in X$. It is clear from Lemma 6 that $\Psi(k)$ is stochastic. Hence, $\|r_{i}(k+1)-s\|\leq V(k)$ for all $i$. In particular, when $\theta_{i}(k)=0$ for all $i$, $\|z_{i}(k+1)-s\|\leq V(k)$ also holds for all $i$ and thus $V(k+1)\leq V(k)$.

Suppose that $\theta_{i}(k)\neq 0$. It is clear that $\sqrt{y_{i}(k)}\geq\|\nabla f_{i}(w_{i}(k))\|^{2}$, and (27)(29) hold under Assumption 3. It follows that

[TABLE]

From the convexity of the function $f_{i}(\cdot)$, $-(w_{i}-s)^{T}\nabla f_{i}(w_{i}(k))\leq f_{i}(s)-f_{i}(w_{i}(k))$. It follows that

[TABLE]

where $s_{i}\in X_{i}$. From the definition of $X_{i}$, $f_{i}(s)-f_{i}(s_{i})\geq 0$ and $f_{i}(s_{i})-f_{i}(w_{i}(k))\leq 0$. Note that $\pi/4\leq\mathrm{arctan}(e^{\|r_{i}(k)\|})\leq\pi/2$ for all $k$ and all $i$. There exists a constant $T_{0}>0$ such that $\frac{kT}{4}<y_{i}(k)<4kT$ for all $k\geq T_{0}$.

[TABLE]

for $k>T_{0}$.

Since $\lim_{\|y\|\rightarrow+\infty}f_{i}(y)=+\infty$ from Lemma 5, there exists one bounded convex closed region $\Omega_{1}=\{y\in\mathbb{R}^{m}\mid\|y-s\|\leq l_{1}\}$ for a constant $l_{1}>0$ such that $f_{i}(w_{i}(k))-f_{i}(s)>4\max_{j}[f_{j}(s)-f_{j}(s_{j})]+4n$ for any $w_{i}(k)\notin\Omega_{1}$. Under Assumption 3, $0<p_{i}(k)T<1$ and hence $1-p_{i}(k)T/2\geq 1/2$. If $w_{i}(k)\in\Omega_{1}$, then it follows from (27) and (40) that $\|z_{i}(k)-s\|\leq 2l_{1}$ and $\|z_{i}(k+1)-s\|^{2}\leq l_{1}^{2}+2/\sqrt{T}+4[f_{i}(s)-f_{i}(s_{i})]/\sqrt{T}\leq l_{1}^{2}+2/\sqrt{T}+4\max_{i}[f_{i}(s)-f_{i}(s_{i})]/\sqrt{T}$. If $\|z_{i}(k)-s\|>2l_{1}$, then we have $w_{i}(k)\notin\Omega_{1}$ and hence $f_{i}(w_{i}(k))-f_{i}(s)\geq 4\max_{i}[f_{j}(s)-f_{j}(s_{j})]+4n$. It follows from (40) that $\|z_{i}(k+1)-s\|\leq\|w_{i}(k)-s\|\leq V(k)$. Summarizing the above analysis, all agents remain in a bounded region. That is, $\|r_{i}(k)\|<\rho$ and $\|z_{i}(k)\|<\rho$ for all $i,k$ and some constant $\rho>0$. Hence, from the definitions of $z_{i}(k)$ and $\sigma_{i}(k)$, it can be obtained that $\|v_{i}(k)\|$ is bounded and hence from the definition of $\sigma_{i}(k)$, $\sigma_{i}(k)$ is lower bounded by a positive constant.

From the definition of $b_{i}(k)$, when $0<p_{i}(k)<\frac{1}{T}$, then $p_{i}(k)\leq b_{i}(k)$. Under Assumption 3, $p_{i}(k)\leq b_{i}(k)\leq p_{i}(k+1)\leq b_{i}(k+1)$ and $p_{i}(k)T<1$. Hence, $1-p_{i}(k)T/2\geq 1/2$, $p_{i}(k)T/2\geq p_{i}(0)T/2$, $1-{b_{i}(k)T}+\frac{p_{i}(k)T}{2}\geq\frac{p_{i}(0)T}{2}$, and $\frac{p_{i}(k)}{p_{i}(k+1)}\geq\frac{p_{i}(0)}{p_{i}(k+1)}\geq p_{i}(0)T$. Consider the matrix $A(k)-\Lambda(k)L(k)\otimes W$. ${b_{i}(k)T}-\frac{p_{i}(k)T}{2}-\sigma_{i}(k)d_{i}T\geq\frac{p_{i}(k)T}{2}-\sigma_{i}(k)d_{i}T\geq\frac{p_{i}(0)T}{2}-d_{i}T>0$. Thus, each nonzero entry of $A(k)-\Lambda(k)L(k)\otimes W$ is lower bounded by a positive constant.

Theorem 3 in [17] and Lemma 1 in [18] both have considered the boundedness of the system states as well but both approaches there cannot be directly applied here because of the different adoption of the interaction mechanism or the lack of the consideration of the time-varying parameters.

Lemma 8**.**

Under Assumptions 1, 2, 3 and 4, there exists a positive integer $h$ and a number $0<\hat{\mu}<1$ such that $\lfloor\Gamma(k_{m+4n}-1,k_{m})\rfloor_{ih}\geq\hat{\mu}$ for all $k_{m}\geq 0$ and $i$.

Proof: The proof is very similar to the proof of Lemma 2 in [18] and hence omitted. It should be noted that each agent might be the root node since the union of the graphs in $[k_{m},k_{m+1}-1)$ is strongly connected under assumption 4.

Lemma 9**.**

Under Assumptions 1, 2, 3 and 4, $\lim_{m\rightarrow+\infty}[\phi_{i}(k)-\phi_{j}(k)]=0$ for all $i,j$, where $\phi_{i}(k),\phi_{j}(k)$ denote the $i,j$th entries of $\phi(k)$.

Proof: Since $\phi(k+1)=\Gamma(k,k)\phi(k)-\nabla F(k)$, then

[TABLE]

Note that $\frac{\pi}{4}\leq\mathrm{arctan}(e^{\|r_{i}(k)\|})\leq\frac{\pi}{2}$. There exists a constant $T_{0}>0$ such that $\frac{kT}{4}<y_{i}(k)<4kT$ for all $k>T_{0}$. Since all $\|r_{i}(k)\|$ and $\|z_{i}(k)\|$ are bounded from Lemma 7 and each $f_{i}(w_{i}(k))$ is differentiable and convex, each $\|w_{i}(k)\|$ is bounded and hence each $\|\nabla f_{i}(w_{i}(k))\|$ is bounded for all $i$. Thus, $\lim_{k\rightarrow+\infty}\|\nabla F(k)\|=0.$ There exists a constant $T_{1}>T_{0}$ for any $\epsilon>0$ such that $\|\nabla F(k)\|<\epsilon$ for all $k>T_{1}$. Since each $\Gamma(k,k)$ is a stochastic matrix from Lemma 6 and $k_{m+1}-1-k_{m}\leq\eta$ under Assumption 4, it follows that

[TABLE]

for $k_{m}>T_{1}$. From Lemma 8, there exists a positive integer $i_{0}$ and a number $0<\hat{\mu}<1$ such that $\lfloor\Gamma(k_{m+4n}-1,k_{m})\rfloor_{ii_{0}}\geq\hat{\mu}$ for all $k_{m}\geq 0$ and all $i$. Note that

[TABLE]

It follows from (43), (46) and (49) that

[TABLE]

and hence

[TABLE]

When $\max_{i}\phi_{i}(k_{m})-\min_{i}\phi_{i}(k_{m})\leq 8n(\eta+1)\epsilon/\hat{\mu}$,

[TABLE]

When $\max_{i}\phi_{i}(k_{m})-\min_{i}\phi_{i}(k_{m})>8n(\eta+1)\epsilon/\hat{\mu}$,

[TABLE]

This means that if $\max_{i}\phi_{i}(k_{m})-\min_{i}\phi_{i}(k_{m})\leq 8n(\eta+1)\epsilon/\hat{\mu}$ holds, then $\max_{i}\phi_{i}(k_{m+4ni})-\min_{i}\phi_{i}(k_{m+4ni})\leq 8n(\eta+1)\epsilon/\hat{\mu}$ for all positive integers $i$. Moreover, note that when $\max_{i}\phi_{i}(k_{m})-\min_{i}\phi_{i}(k_{m})>16n(\eta+1)\epsilon/\hat{\mu}$,

[TABLE]

This means that all $\phi_{i}(k_{m+4n})$ will converge to the region in finite time where

[TABLE]

In view of the arbitrariness of the adoption of $k_{m}$ and $\epsilon$, letting $\epsilon\rightarrow 0$, it follows that $\lim_{m\rightarrow+\infty}[\phi_{i}(k_{m})-\phi_{j}(k_{m})]=0$ for all $i,j$. Note that $\|\phi(k+1)-\phi(k_{m})\|\leq\|\Gamma(k,k_{m})\phi(k_{m})-\phi(k_{m})\|+4n(\eta+1)\epsilon$ for all $k_{m}<k\leq k_{m+4n}-1$ and $\Gamma(k,k_{m})$ is stochastic. Letting $k_{m}\rightarrow+\infty$ and $\epsilon\rightarrow 0$, it follows that $\lim_{k\rightarrow+\infty}\|\phi(k+1)-\phi(k_{m})\|=\lim_{k\rightarrow+\infty}[\phi_{i}(k)-\phi_{j}(k)]=0$ for all $i,j$.

Lemma 9 is a key lemma to study the optimal convergence of the system (3) with (18). In contrast to Lemma 3 in [18], Lemma 9 need consider not only the interaction between agents but also the effects of the gradient term, which makes the analysis much more complicated.

Lemma 10**.**

For the system given by ${y}_{i}(k+1)={y}_{i}(k)+\mathrm{arctan}(e^{\|r_{i}(k)\|})T$ with $y_{i}(0)>0$, if $\lim_{k\rightarrow+\infty}[r_{i}(k)-r_{j}(k)]=0$ for all $i,j$, $\lim_{k\rightarrow+\infty}\frac{y_{i}(k)}{y_{j}(k)}=1$ for all $i,j$.

Proof: Note that $\pi/4\leq\mathrm{arctan}(e^{\|r_{i}(k)\|})\leq\pi/2$ for all $i$ and all $k$. There exists an integer $T_{0}>0$ such that $\frac{kT}{4}<y_{i}(k)<4kT$ for all $k>T_{0}$ and all $i$. Let $\Delta_{i}(k)=y_{i}(0)-y_{1}(0)+\sum_{s=0}^{k}[\mathrm{arctan}(e^{\|r_{i}(s)\|})-\mathrm{arctan}(e^{\|r_{1}(s)\|})]T$. It is clear that $y_{i}(k)=y_{1}(k)(1+\Delta_{i}(k)/y_{1}(k))$ for all $i$. Since $\lim_{k\rightarrow+\infty}[r_{i}(k)-r_{j}(k)]=0$, from the continuity of the function $\mathrm{arctan}(e^{\|r_{i}(s)\|})$, there exists an integer $T_{1}>T_{0}$ for any ${\epsilon}_{0}>0$ such that $|\mathrm{arctan}(e^{\|r_{i}(k)\|})-\mathrm{arctan}(e^{\|r_{1}(k)\|})|<{\epsilon}_{0}$ for all $k>T_{1}$ and all $i$. It is clear that $\lim_{k\rightarrow+\infty}\Delta_{i}(T_{1})/y_{1}(k)=0$ and $|\sum_{s=T_{1}+1}^{k}[\mathrm{arctan}(e^{\|r_{i}(s)\|})-\mathrm{arctan}(e^{\|r_{1}(s)\|})]T/y_{1}(k)|$ $<4{\epsilon}_{0}$. Since ${\epsilon}_{0}$ can be arbitrarily chosen, we have $\lim_{k\rightarrow+\infty}\Delta_{i}(k)/y_{1}(k)=\lim_{k\rightarrow+\infty}[\Delta_{i}(T_{1})+\sum_{s=T_{1}+1}^{k}[\mathrm{arctan}(e^{\|x_{i}(s)\|})-\mathrm{arctan}(e^{\|x_{1}(s)\|})]]/y_{1}(k)=0$ and hence $\lim_{k\rightarrow+\infty}\frac{y_{i}(k)}{y_{1}(k)}=1$ for all $i$. Therefore, $\lim_{k\rightarrow+\infty}\frac{y_{i}(k)}{y_{j}(k)}=1$ for all $i,j$.

Assumption 5**.**

Each graph $\mathcal{G}(k)$ is balanced, i.e., $\sum_{j=1}^{n}a_{ij}(k)=\sum_{j=1}^{n}a_{ji}(k)$.

The role of Assumption 5 is to balance the rate of the optimal convergence of all local objective functions.

Theorem 1**.**

Under Assumptions 1, 2, 3, 4 and 5, using (18) for (3), all agents reach a consensus and minimize the team objective function (7) as $k\rightarrow+\infty$. **

Proof: From Lemma 9, $\lim_{k\rightarrow+\infty}[r_{i}(k)-z_{j}(k)]=0$ for all $i,j$. From the definition of $z_{i}(k)$, $\lim_{k\rightarrow+\infty}v_{i}(k)=0$ for all $i$. There exists a constant $T_{0}>0$ such that $\|v_{i}(k)\|<\underline{\rho}_{i}$ for all $i$ and all $k>T_{0}$. This means $\sigma_{i}(k)=1$ and hence $p_{i}(k)=b_{i}(k)$ and $p_{i}(k)=p_{j}(k+1)$ for all $i$ and $k>T_{0}$. As a result, the equations (22), (27) and (29) hold for $k>T_{0}$. Let $\phi^{*}(k)=\frac{1}{2n}\sum_{i=1}^{2n}\phi_{i}(k)$. Under Assumption 5, it can be checked that $\textbf{1}^{T}\Psi(k)=\textbf{1}^{T}$ for all $k$, where $\mathbf{1}$ denotes a column vector of all ones with a compatible dimension. It follows from (31) that

[TABLE]

Consider the Lyapunov function candidate $V_{1}(k)=\|\phi^{*}(k)-P_{X}(\phi^{*}(k))\|^{2}$ for $k>T_{0}$. From Lemma 1, $[\phi^{*}(k+1)-P_{X}(\phi^{*}(k+1))]^{T}[P_{X}(\phi^{*}(k))-P_{X}(\phi^{*}(k+1))]\leq 0$. This implies that the angle between the vectors $\phi^{*}(k+1)-P_{X}(\phi^{*}(k+1))$ and $P_{X}(\phi^{*}(k))-P_{X}(\phi^{*}(k+1))$ lies in $[\pi/2,\pi]$. From the triangle relationship, the angle between the vectors $\phi^{*}(k+1)-P_{X}(\phi^{*}(k))$ and $P_{X}(\phi^{*}(k))-P_{X}(\phi^{*}(k+1))$ also lies in $[\pi/2,\pi]$. That is, $[\phi^{*}(k+1)-P_{X}(\phi^{*}(k))]^{T}[P_{X}(\phi^{*}(k))-P_{X}(\phi^{*}(k+1))]\leq 0$. It follows from Lemma 1 and (57) that

[TABLE]

From Lemma 7, $\|r_{i}(k)\|$ and $\|z_{i}(k)\|$ are bounded for all $i,k$. Hence all $\|w_{i}(k)\|$ and $\phi^{*}(k)$ are bounded for all $i,k$. Since $f_{i}(w_{i}(k))$ is differentiable and convex, $\|\nabla f_{i}(w_{i}(k))\|$ is also bounded for all $i,k$. Note that $\frac{\pi}{4}\leq\mathrm{arctan}(e^{\|r_{i}(k)\|})\leq\frac{\pi}{2}$. There exists a constant $T_{1}>0$ such that $\frac{kT}{4}<y_{i}(k)<4kT$ for all $k>T_{1}$. It follows from (57) that

[TABLE]

for all $k>T_{1}$ and some constant $\gamma>0$. Note that $\lim_{k\rightarrow+\infty}[\phi_{i}(k)-\phi_{j}(k)]=0$ from Lemma 9, $\frac{kT}{4}<y_{i}(k)<4kT$ for all $k>T_{1}$ and each function $f_{i}(\cdot)$ is differentiable. Then $\lim_{k\rightarrow+\infty}[w_{i}(k)-\phi^{*}(k)]=0$ for all $i$ and there exists a constant $T_{2}>T_{1}$ for any $\epsilon_{1}>0$ such that $|w_{i}(k)-\phi^{*}(k)|<\epsilon_{1}$, $|\phi_{i}(k)-\phi^{*}(k)|<\epsilon_{1}$, and $|f_{i}(w_{i}(k))-f_{i}(\phi^{*}(k))|<\epsilon_{1}$ for all $k>T_{2}$. From the definition of the projection operator, we have $\|\phi^{*}(k)-P_{X}(\phi^{*}(k))\|\leq\|\phi^{*}(k)-P_{X}(\phi^{*}(k-1))\|$. Thus,

[TABLE]

for $k>T_{2}$ and some constant $\tilde{\gamma}>0$, where the third inequality has used the convexity of $f_{i}(w_{i}(k))$.

From Lemma 10, $\lim_{k\rightarrow+\infty}\frac{y_{i}(k)}{y_{j}(k)}=1$ for all $i,j$. Together with the analysis above (71), there must exist a constant $T_{3}>T_{2}$ such that {\Big{|}}1-\sqrt{\frac{y_{i}(k)}{y_{j}(k)}}{\Big{|}}<\epsilon_{1} and {\Big{|}}(\frac{\sqrt{y_{1}(k)}}{\sqrt{y_{i}(k)}}-1)[f_{i}(\phi^{*}(k))-f_{i}(P_{X}(\phi^{*}(k)))]{\Big{|}}<\epsilon_{1} for all $k>T_{3}$. Let $T_{3}$ be sufficiently large such that $\frac{1}{\sqrt{k}}<\epsilon_{1}$ for all $k>T_{3}$. It follows that

[TABLE]

for all $k>T_{3}$ and some constant $\hat{\gamma}>0$.

Let $\Omega_{1}=\{x\mid\|x-P_{X}(x)\|\leq l_{1}\}$ and $\Omega_{2}=\{x\mid\|x-P_{X}(x)\|\leq l_{2}\}$ for two constants $l_{1}>0$ and $l_{2}>0$ be two sets such that $l_{2}>l_{1}+2\epsilon_{1}$ and $\min_{x\in\bar{\partial}\Omega_{1}}\sum_{i=1}^{n}[f_{i}(x)-f_{i}(s)]=\hat{\gamma}\epsilon_{1}+2\epsilon_{1}$ for any $s\in X$ where $\bar{\partial}\Omega_{1}$ denotes the boundary of $\Omega_{1}$. It is clear from Lemma 4 that $\sum_{i=1}^{n}(f_{i}(s_{1})-f_{i}(s))>\hat{\gamma}\epsilon_{1}+2\epsilon_{1}$ for any $s_{1}\notin\Omega_{1}$. Note that each $\|\nabla f_{i}(w_{i}(k))\|$ is bounded. Let $T_{3}$ be further sufficiently large such that $\|\nabla F(k)\|<\epsilon_{1}$ for all $k>T_{3}$. From (57), $\|\phi^{*}(k+1)-\phi^{*}(k)\|\leq\|\frac{1}{2n}\mathbf{1}^{T}\nabla F(k)\|<\epsilon_{1}$ for all $k>T_{3}$. From the definition of $\Omega_{2}$, it is easy to see that for $k>T_{3}$, $\phi^{*}(k+1)\in\Omega_{2}$ when $\phi^{*}(k)\in\Omega_{1}$. When $\phi^{*}(k)\notin\Omega_{1}$ and $\phi^{*}(k)\in\Omega_{2}$, $V_{1}(k+1)-V_{1}(k)<0$ and hence $\|\phi^{*}(k+1)-P_{X}(\phi^{*}(k+1))\|\leq\|\phi^{*}(k)-P_{X}(\phi^{*}(k))\|$ for $k>T_{3}$, implying that $\phi^{*}(k+1)\in\Omega_{2}$. By induction, it follows that if $\phi^{*}(\tilde{k})\in\Omega_{2}$ for some $\tilde{k}>T_{3}$, then $\phi^{*}({k})\in\Omega_{2}$ for all $k>\tilde{k}$. When $\phi^{*}(k)\notin\Omega_{2}$ for $k>T_{3}$, $V_{1}(k+1)-V_{1}(k)<-\frac{1}{n\sqrt{kT}}\epsilon_{1}$. Since $\sum_{k=h}^{+\infty}\frac{1}{\sqrt{kT}}=+\infty$ for some positive constant $h>0$, there exists a constant $T_{4}>T_{3}$ such that $\phi^{*}(k)\in\Omega_{2}$ for all $k>T_{4}$. Since $\epsilon_{1}$ can be arbitrarily chosen, letting $\epsilon_{1}\rightarrow 0$, it follows from Lemma 4 and the definitions of $\Omega_{1}$ and $\Omega_{2}$ that $\lim_{k\rightarrow+\infty}\sum_{i=1}^{n}(f_{i}(\phi^{*}(k))-f_{i}(s))=0$. That is, the team objective function (7) is minimized as $k\rightarrow+\infty$.

VI Extension to the case with nonuniform convex position constraints

In some applications, besides the velocity constraints, each agent’s position state might also be constrained to a certain area. In this section, we extend the results in previous sections to the case where there further exist nonuniform position constraints. Here it is assumed that each agent’s position state remains in a closed convex set, denoted by $H_{i}$, which is known to only agent $i$. Under this circumstance, each agent has the following dynamics:

[TABLE]

where $v_{i}$ is subject to the nonconvex constraint set $V_{i}$ as in previous sections. The Problem (7) now becomes

[TABLE]

To solve this propblem in a distributed manner, we propose the algorithm given by

[TABLE]

for all $k\geq 0$, where $r_{i}(0)=P_{H_{i}}[r_{i}(0)]$, $v_{i}(0)=S_{V_{i}}(v_{i}(0))$ and $p_{i}>0$ is the feedback damping gain. Here, the parameters $p_{i}$ are assumed to be constant for easy readability of our results.

Compared with (18), the switching mechanism in (88) is simplified. This is because all agents’ states are bounded under Assumption 1 and the the following assumption and there is no need to introduce switching rules to ensure the boundedness of the agents’ states.

Assumption 6**.**

Each $H_{i}$ is closed and bounded for all $i$ and there exists a scalar $\delta>0$ and a vector $\bar{x}\in H$ such that $\{\xi|\|\xi-\bar{x}\|\leq\delta\}\subset H$.

Assumption 6 ensures that $H$ contains at least an interior point.

To present our main theorem under this situation, we need to modify Assumption 3 as follows.

Assumption 7**.**

Suppose that $\frac{1}{T}>p_{i}>0$ for all $i$, and there exist a constant $d_{i}>0$ such that $p_{i}>2d_{i}>2\lfloor{L}({k})\rfloor_{ii}$ for all $i$.

Theorem 2**.**

Under Assumptions 1, 4, 5, 6 and 7, using (88) for (75), all agents reach a consensus and minimize the team objective function (7) as $k\rightarrow+\infty$. **

Sketch of Proof: Let $\sigma_{i}(k),z_{i}(k)$ and $b_{i}(k)$ be defined as previously. Specially, $z_{i}(k)=r_{i}(k)+\frac{2}{p_{i}}v_{i}(k)$. Under Assumption 6, all $r_{i}(k)$ are bounded. From the definition of $z_{i}(k)$, under Assumption 7, it can be proved that all $z_{i}(k)$ are bounded and all $\sigma_{i}(k)$ are lower bounded by a positive constant. For some $i$, let $0\leq c_{i}(k)\leq 1$ be a scaling factor such that $\varphi_{h}(k)\triangleq r_{i}(k)+c_{i}(k)\frac{p_{i}}{2}[z_{i}(k)-r_{i}(k)]T\in H_{i}$ and $r_{i}(k)+\frac{cp_{i}}{2}[z_{i}(k)-r_{i}(k)]T\notin H_{i}$ for all $1\geq c>c_{i}(k)$. Suppose that $c_{i}(k)<1$. Let $\varphi_{g}(k)=r_{i}(k)+\frac{p_{i}}{2}[z_{i}(k)-r_{i}(k)]T$, $\varphi_{p}(k)=P_{H_{i}}[\varphi_{g}(k)]$ and $\varphi_{i}(k)=\varphi_{p}(k)-\varphi_{h}(k)$. Then by simple calculations similar to (25), we have

[TABLE]

where $\varphi_{pz}(k)=\varphi_{c}(k)+\frac{1}{c_{s}}\varphi_{i}(k)$, $\varphi_{c}(k)=c_{z}(k)z_{i}(k)+\sum_{j\in\{i\cup\mathcal{N}_{i}(k)\}}c_{jr}(k)r_{i}(k)$, and $c_{s}(k)=\frac{c_{ir}(k)}{\sum_{j\in\{i\cup\mathcal{N}_{i}(k)\}}c_{jr}(k)}$. It is clear that $c_{z}(k)+\sum_{j\in\{i\cup\mathcal{N}_{i}(k)\}}c_{jr}(k)=c_{s}(k)+(1-c_{s}(k))c_{z}(k)+\sum_{j\in\mathcal{N}_{i}(k)}c_{jr}(k)=1$ and $\sum_{j\in\{i\cup\mathcal{N}_{i}(k)\}}c_{jr}(k)=b_{i}(k)T-c_{i}(k)p_{i}T/2\geq b_{i}(k)T-p_{i}T/2$. Under Assumption 7, $b_{i}(k)\geq p_{i}$ and $b_{i}(k)T<1$ as previously discussed. Hence $c_{ir}(k)\geq b_{i}(k)T-c_{i}(k)p_{i}T/2-d_{i}T/2\geq d_{i}T/2$ and $0<c_{z}(k)=1-b_{i}(k)T+c_{i}(k)p_{i}T/2\leq 1-b_{i}(k)T+p_{i}T/2\leq 1-p_{i}T/2$. Note that $c_{jr}(k)=a_{ij}(k)/2\geq 0$. The coefficients of $z_{i}(k)$, $r_{i}(k)$ and $r_{j}(k)$ in (96) are all nonnegative. From the definition of the projection operator and Lemma 2 in [8], $\|z_{i}(k+1)-P_{H}(z_{i}(k+1))\|\leq\|z_{i}(k+1)-P_{H}(z_{i}(k+1)+\frac{2\sigma_{i}(k)}{p_{i}}\theta_{i}(k))\|\leq\|z_{i}(k+1)+\frac{2\sigma_{i}(k)}{p_{i}}\theta_{i}(k)-P_{H}(z_{i}(k+1)+\frac{2\sigma_{i}(k)}{p_{i}}\theta_{i}(k))\|+\|\frac{2\sigma_{i}(k)}{p_{i}}\theta_{i}(k)\|\leq c_{s}(k)\|\varphi_{pz}(k)-P_{H}(\varphi_{pz}(k))\|+\|\frac{2\sigma_{i}(k)}{p_{i}}\theta_{i}(k)\|+(1-c_{s}(k))c_{z}(k)\|z_{i}(k)-P_{H}(z_{i}(k))\|+\sum_{j\in\mathcal{N}_{i}(k)}c_{jr}(k)\|r_{j}(k)-P_{H}(r_{j}(k))\|$.

In the following, we study the term $\|\varphi_{pz}(k)-P_{H}(\varphi_{pz}(k))\|$ to analyze the convergence of $z_{i}(k)$. Note that $\sum_{j\in\mathcal{N}_{i}(k)}c_{jr}(k)\leq d_{i}T/2$, $\sum_{j\in\{i\cup\mathcal{N}_{i}(k)\}}c_{jr}(k)\geq b_{i}(k)T-p_{i}T/2\geq p_{i}T/2$ and $p_{i}>2d_{i}$ under Assumption 7. Thus, $1/2<\frac{p_{i}-d_{i}}{p_{i}}\leq c_{s}(k)\leq 1$. Let $\varphi_{z}(k)=\varphi_{c}(k)+(1-c_{i}(k))(z_{i}(k)-r_{i}(k))p_{i}T/2/c_{s}(k)\triangleq z_{i}(k)(1-c_{d}(k))+r_{i}(k)c_{d}(k)$, where $c_{d}(k)=b_{i}(k)T-\frac{c_{i}(k)p_{i}T}{2}-(1-c_{i}(k))p_{i}T/2/c_{s}(k)$. It is clear that the triangles formed by the points $\varphi_{c}(k)$, $\varphi_{pz}(k)$ and $\varphi_{z}(k)$, and formed by $\varphi_{h}(k)$, $\varphi_{p}(k)$ and $\varphi_{g}(k)$ are similar. Moreover, note from (96) that $\varphi_{c}(k)=z_{i}(k)+[\frac{c_{i}(k)p_{i}T}{2}-b_{i}(k)T][z_{i}(k)-r_{i}(k)]$. Since $c_{i}(k)\leq 1$ and $b_{i}(k)\geq p_{i}$, then $b_{i}(k)T-\frac{c_{i}(k)p_{i}T}{2}\geq\frac{c_{i}(k)p_{i}T}{2}$ and hence $\varphi_{c}(k)\notin H_{i}$ based on the definition of $c_{i}(k)$ and the convexity of the convex set $H_{i}$. Let $\Gamma$ be a hyperplane such that $\varphi_{z}(k)-\varphi_{pz}(k)\perp\Gamma$ and $\varphi_{p}(k)\in\Gamma$. It is clear that $\varphi_{g}(k)-\varphi_{p}(k)\perp\Gamma$ and all points of the convex set $H_{i}$ lie on one side of the hyperplane $\Gamma$. By considering the relationship between the aforementioned similar triangles, it can be obtained that $\varphi_{z}(k)$ and $\varphi_{pz}(k)$ lie on the other side of $\Gamma$. Recall that $\varphi_{z}(k)-\varphi_{pz}(k)\perp\Gamma$. The angle between the vectors $P_{H}(\varphi_{z}(k))-\varphi_{pz}(k)$ and $\varphi_{z}(k)-\varphi_{pz}(k)$ is no smaller than $\pi/2$. Thus, $\|\varphi_{pz}(k)-P_{H}(\varphi_{z}(k))\|\leq\|\varphi_{z}(k)-P_{H}(\varphi_{z}(k))\|$. On the other hand, from Lemma 2 in [8], $\|\varphi_{pz}(k)-P_{H}(\varphi_{pz}(k))\|\leq\|\varphi_{pz}(k)-P_{H}(\varphi_{z}(k))\|\leq\|\varphi_{z}(k)-P_{H}(\varphi_{z}(k))\|\leq(1-c_{d}(k))\|z_{i}(k)-P_{H}(z_{i}(k))\|+c_{d}(k)\|r_{i}(k)-P_{H}(r_{i}(k))\|$. It should be noted here that since $1\leq 1/c_{s}\leq\frac{p_{i}}{p_{i}-d_{i}}<2$ and hence it can be proved that $\rho_{c}<c_{d}(k)\leq b_{i}(k)T-p_{i}T/2$ for some constant $\rho_{c}>0$.

Summarizing the analysis above, it can be proved that $\|z_{i}(k+1)-P_{H}(z_{i}(k+1))\|\leq\tilde{c}_{z}(k)\|z_{i}(k)-P_{H}(z_{i}(k))\|+\|\frac{2\sigma_{i}(k)}{p_{i}}\theta_{i}(k)\|+\tilde{c}_{ir}(k)\|r_{i}(k)-P_{H}(r_{i}(k))\|+\sum_{j\in\mathcal{N}_{i}(k)}c_{jr}(k)\|r_{j}(k)-P_{H}(r_{j}(k))\|$ where $\tilde{c}_{z}(k)=c_{s}(k)(1-c_{d}(k))+(1-c_{s}(k))c_{z}(k)$, $\tilde{c}_{ir}(k)=c_{s}(k)c_{d}(k)$ and $c_{jr}(k)$ are lower bounded by a common positive constant, and $\tilde{c}_{z}(k)+\tilde{c}_{ir}(k)+\sum_{j\in\mathcal{N}_{i}(k)}c_{jr}(k)=1$ for all $j,k$. For the case of $c_{i}(k)=1$, similar statements can be obtained in a similar way. Following the lines of the proofs of Lemmas 9, 11, 12 and Proposition 1 in [15], it can be proved that $\lim_{k\rightarrow+\infty}[r_{i}(k)-z_{j}(k)]=0$ for all $i,j$. Further, by a similar analysis in the proof of Theorem 1, it can be proved that the team objective function (78) is minimized as $k\rightarrow+\infty$.

Remark 2**.**

In our previous work [16], nonuniform stepsizes were considered but the analysis approach is hard to be applied directly here because this note is different in nature from [16] in four aspects. First, the constraint sets considered in [16] are uniform and convex while the constraint sets considered in this note are nonuniform and some might be nonconvex. Different convex constraint sets and nonconvex constraint sets might yield different nonlinearities. The coupling of different convex constraint sets and nonconvex constraint sets would yield more complicated nonlinearities. Second, the communication graphs in [16] are assumed to be strongly connected at each time while the graphs in this note are assumed to be jointly strongly connected, which is more general and also much harder to analyze. Third, the agent dynamics is in the form of single integrators in [16] while the agent dynamics is in the form of double integrators here in this note. Fourth, in [16], sign functions are used for the interactions between agents which can compensate for the inconsistent local gradients between neighbors and make the design and analysis relatively easier. In contrast, in this note, a kind of linear continuous consensus functions are used instead and no sign functions are used, which also makes the analysis of the system different from that in [16].

VII Numerical Examples

Consider a multi-agent system with 8 agents in $\mathbb{R}^{2}$. The communication graphs switch among the balanced subgraphs of the graph shown in Fig. 1. Each edge weight is 0.5. The sample time is $T=0.2$ $s$ and the union of the communication graphs every $10$ $s$ is strongly connected. The local objective functions for the agents are $f_{1}(r_{1})=(r_{11}-1)^{2}+(r_{12}-1)^{2}$, $f_{2}(r_{2})=r_{21}^{2}+r_{22}^{2}$, $f_{3}(r_{3})=(r_{31}-1)^{4}+(r_{32}-1)^{4}$, $f_{4}(r_{4})=r_{41}^{4}+r_{42}^{4}$, $f_{5}(r_{5})=(r_{51}-1)^{2}+r_{52}^{2}$, $f_{6}(r_{6})=r_{61}^{2}+(r_{62}-1)^{2}$, $f_{7}(r_{7})=(r_{71}-1)^{4}+r_{72}^{4}$, and $f_{8}(r_{8})=r_{81}^{4}+(r_{82}-1)^{4}$. The team performance function (7) is minimized if and only if $r=[0.5,0.5]^{T}$. The velocity constraint set of each agent is $V_{i}=\{v\mid\|v\|\leq 1\}\cup\{v\mid-0.5\leq[1,0]^{T}v\leq 0.5,0\leq[0,1]^{T}v\leq 1.5\}$ for all $i$. The position constraint set for agents $1,2,3,4$ are $H_{1}=\{r\mid\|r\|\leq 1\}$ while for agents $5,6,7,8$ is $H_{2}=\{r=[r_{a},r_{b}]^{T}\mid-6.5\leq r_{a}\leq-0.5,-3\leq r_{b}\leq 3\}$. The team performance function (78) is minimized if and only if $r=[-0.5,0.5]^{T}$. According to Assumption 3, $p_{i}(0)$ is taken as $p_{i}(0)=1.5$ for algorithm (18). Fig. 2 shows the simulation results with nonconvex velocity constraints and nonuniform stepsizes. It is clear that all agents reach a consensus and minimize the team performance function (7) while their velocities remaining in $V_{i}$, which is consistent with Theorem 1. According to Assumption 7, $p_{i}$ is taken as $p_{i}=1.5$ for algorithm (88). Fig. 3 show the simulation results with nonconvex velocity constraints, nonuniform position constraints and nonuniform stepsizes. It is clear that all agents reach a consensus and minimize the team performance function (78) while their positions and velocities remaining in their corresponding constraint sets $H_{i}$ and $V_{i}$, which is consistent with Theorem 2 as well.

VIII Conclusions

In this note, a distributed optimization problem of multi-agent systems with nonconvex velocity constraints, nonuniform position constraints and nonuniform stepsizes was studied. Two distributed constrained algorithms were proposed. The algorithm gains need not to be predesigned and can be given by each agent using its own and neighbours’ information. The system considered was a nonlinear time-varying one and the analysis was performed based on a model transformation and the properties of stochastic matrices. It was shown that the optimization problem can be solved as long as the union of the communication graphs among each certain interval are jointly strongly connected and balanced.