Distributed Optimization for a Class of High-order Nonlinear Multi-agent Systems with Unknown Dynamics

TL;DR

This paper develops distributed adaptive control algorithms for high-order nonlinear multi-agent systems with unknown dynamics, enabling agents to collaboratively reach the global optimum despite nonlinear uncertainties.

Contribution

It introduces a novel embedded control design and adaptive controllers for high-order nonlinear agents, addressing unknown dynamics in distributed optimization.

Findings

Agents' outputs converge to the global optimal solution.

Estimated parameters converge to true values under persistence of excitation.

Algorithms are validated through simulation examples.

Abstract

In this paper, we study a distributed optimization problem for a class of high-order multi-agent systems with unknown dynamics. In comparison with existing results for integrators or linear agents, we need to overcome the difficulties brought by the unknown nonlinearities and also the optimization requirement. For this purpose, we employ an embedded control based design and first convert this problem into an output stabilization problem. Then, two kinds of adaptive controllers are given for these agents to drive their outputs to the global optimal solution under some mild conditions. Finally, we show that the estimated parameter vector converges to the true parameter vector under some well-known persistence of excitation condition. The efficacy of these algorithms was verified by a simulation example.

Equations

$$\begin{array}{rl}{\dot{x}}_{j,i}& =x_{j+1,i}\\ {\dot{x}}_{n_i,i}& ={\Delta }_i(x_i,{\theta }_i,t)+u_i\\ y_i& =x_{1,i},i=1,\dots ,N,j=1,\dots ,n_i-1\end{array}$$

$$y^{\ast }=\arg \underset{y\in \mathbb{R}}{\min }f(y)={\sum }_{i=1}^N{f}_i(y)$$

$$\underset{t\to +\infty }{\lim }||y_i(t)-y^{\ast }||=0,i=1,\dots ,N$$

$$\begin{array}{rl}{\dot{r}}_i& =-\nabla f_i(r_i)-{\sum }_{j=1}^N{a}_{ij}({\lambda }_i-{\lambda }_j)\\ {\dot{\lambda }}_i& ={\sum }_{j=1}^N{a}_{ij}(r_i-r_j)\end{array}$$

$$\begin{array}{rl}{u}_i& =-{\hat{\theta }}_i^{\mathrm{T}}{\mathbf{p}}_i(x_i,t)+\frac{1}{{\epsilon }^{n_i}}[k_{1i}(x_{1,i}-r_i)+{\sum }_{j=2}^{n_i}{\epsilon }^{j-1}{k}_{ji}{x}_{j,i}]\\ {\dot{\hat{\theta }}}_i& ={\varphi }_i(x_i,{\hat{\theta }}_i,r_i,t)\\ {\dot{r}}_i& =-\nabla f_i(y_i)-{\sum }_{j=1}^N{a}_{ij}({\lambda }_i-{\lambda }_j)\\ {\dot{\lambda }}_i& ={\sum }_{j=1}^N{a}_{ij}(r_i-r_j)\end{array}$$

$$\begin{array}{rl}{\dot{x}}_{1,i}& =x_{2,i}\\ ⋮& \\ {\dot{x}}_{n_i,i}& =({\theta }_i^{\mathrm{T}}-{\hat{\theta }}_i^{\mathrm{T}}){\mathbf{p}}_i(x_i,t)+\frac{1}{{\epsilon }^{n_i}}[k_{1i}(x_{1,i}-r_i)+{\sum }_{j=2}^{n_i}{\epsilon }^{j-1}{k}_{ji}{x}_{j,i}]\\ {\dot{\hat{\theta }}}_i& ={\varphi }_i(x_i,{\hat{\theta }}_i,r_i,t)\\ {\dot{r}}_i& =-\nabla f_i(y_i)-{\sum }_{j=1}^N{a}_{ij}({\lambda }_i-{\lambda }_j)\\ {\dot{\lambda }}_i& ={\sum }_{j=1}^N{a}_{ij}(r_i-r_j)\end{array}$$

$$\begin{array}{rl}\epsilon {\dot{\hat{x}}}_i& =A_i{\hat{x}}_i-\epsilon b_{1i}{\dot{r}}_i+{\epsilon }^{n_i}{b}_{2i}({\theta }_i^{\mathrm{T}}-{\hat{\theta }}_i^{\mathrm{T}}){\mathbf{p}}_i(x_i,t)\\ {\dot{\hat{\theta }}}_i& ={\varphi }_i(x_i,{\hat{\theta }}_i,r_i,t)\\ {\dot{r}}_i& =-\nabla f_i(y_i)-{\sum }_{j=1}^N{a}_{ij}({\lambda }_i-{\lambda }_j)\\ {\dot{\lambda }}_i& ={\sum }_{j=1}^N{a}_{ij}(r_i-r_j)\end{array}$$

$$\begin{array}{rl}\dot{\hat{x}}& =\frac{1}{\epsilon }A\hat{x}-B_1\dot{r}+E{B}_2{\mathbf{p}}^{\mathrm{T}}(x,t)\bar{\theta }\\ \dot{\bar{\theta }}& =\varphi (x,\hat{\theta },r,t)\\ \dot{r}& =-\nabla \tilde{f}(y)-L\lambda \\ \dot{\lambda }& =Lr\end{array}$$

$$A_i^{\mathrm{T}}{P}_i+P_i{A}_i=-2I_{n_i}$$

$${\dot{\hat{V}}}_i=$$

$$-2{\epsilon }^{n_i-1}{\bar{\theta }}_i^{\mathrm{T}}{\varphi }_i(x_i,{\hat{\theta }}_i,r_i,t)$$

$$=$$

$${\dot{\hat{V}}}_i\le$$

$$\dot{\hat{V}}\le$$

$$\dot{r}$$

$$\dot{\lambda }$$

$$\dot{\bar{r}}$$

$$\dot{\bar{\lambda }}$$

$$\dot{\bar{r}}=-\mathbf{h}(r,r^{\ast })-LR{\hat{\lambda }}_2+\mathbf{h}(r,y),{\dot{\hat{\lambda }}}_2=R^{\mathrm{T}}L\bar{r}$$

$$c_3||\hat{r}|{|}^2\le \bar{V}(\hat{r})\le c_4||\hat{r}|{|}^2$$

$$\frac{\partial \bar{V}}{\partial \bar{r}}[-\mathbf{h}(r,r^{\ast })-LR{\hat{\lambda }}_2]+\frac{\partial \bar{V}}{\partial {\bar{\lambda }}_2}{R}^{\mathrm{T}}L\bar{r}\le -c_5||\hat{r}|{|}^2$$

$$||\frac{\partial \bar{V}}{\partial \hat{r}}||\le c_6||\hat{r}||$$

$$\begin{array}{rl}\dot{\bar{V}}=& \frac{\partial \bar{V}}{\partial \bar{r}}[-\mathbf{h}(r,r^{\ast })-LR{\hat{\lambda }}_2]+\frac{\partial \bar{V}}{\partial {\bar{\lambda }}_2}{R}^{\mathrm{T}}L\bar{r}+\frac{\partial \bar{V}}{\partial \hat{r}}\mathbf{h}(r,y)\\ & \le -c_5||\hat{r}|{|}^2+c_6\bar{l}||\hat{r}||||\hat{x}||\end{array}$$

$$\dot{V}\le$$

$$\le -(\frac{c_1}{\epsilon }-c_2{c}_7-c^2)||\hat{x}|{|}^2-(c{c}_5-c_2{c}_7-c_6^2{\bar{l}}^2)||\hat{r}|{|}^2$$

$$\dot{V}\le$$

$$||y_i(t)-y^{\ast }||\le ||x_{1,i}-r_i(t)||+||r_i(t)-y^{\ast }||\to 0$$

$$\begin{array}{rl}{u}_i& =-{\hat{\theta }}_i^{\mathrm{T}}{\mathbf{p}}_i(x_i,t)+\frac{1}{{\epsilon }^{n_i}}[k_{1i}(x_{1,i}-r_i)+{\sum }_{j=2}^{n_i}{\epsilon }^{j-1}{k}_{ji}{x}_{j,i}]\\ {\dot{\hat{\theta }}}_i& ={\mathbf{p}}_i(x_i,t)b_{2i}^{\mathrm{T}}{P}_i{\hat{x}}_i\\ {\dot{r}}_i& =-\nabla f_i(r_i)-{\sum }_{j=1}^N{a}_{ij}({\lambda }_i-{\lambda }_j)\\ {\dot{\lambda }}_i& ={\sum }_{j=1}^N{a}_{ij}(r_i-r_j)\end{array}$$

$$\begin{array}{rl}{u}_i& =-{\hat{\theta }}_i^{\mathrm{T}}{\mathbf{p}}_i(x_i,t)+\frac{1}{{\epsilon }^{n_i}}[k_{1i}(x_{1,i}-r_i)+{\sum }_{j=2}^{n_i}{\epsilon }^{j-1}{k}_{ji}{x}_{j,i}]\\ {\dot{\hat{\theta }}}_i& =-{\sigma }_{{\theta }_i}{\hat{\theta }}_i+{\mathbf{p}}_i(x_i,t)b_{2i}^{\mathrm{T}}{P}_i{\hat{x}}_i\\ {\dot{r}}_i& =-\nabla f_i(y_i)-{\sum }_{j=1}^N{a}_{ij}({\lambda }_i-{\lambda }_j)\\ {\dot{\lambda }}_i& ={\sum }_{j=1}^N{a}_{ij}(r_i-r_j)\end{array}$$

$$\frac{1}{T_0}{\int }_t^{t+T_0}{\mathbf{p}}_i(x_i(\tau ),\tau ){\mathbf{p}}^{\mathrm{T}}(x_i(\tau ),\tau )\mathrm{d}\tau \ge m{I}_{n_{{\theta }_i}},\forall t\ge t_0$$

$${\dot{x}}_{1,i}$$

$${\dot{x}}_{2,i}$$

$$y_i$$

$$d_i(t)=D_i{v}_i(t),{\dot{v}}_i=S_i{v}_i$$

Full text

Distributed Optimization for a Class of High-order Nonlinear Multi-agent Systems with Unknown Dynamics

Yutao Tang ***This work was supported by National Natural Science Foundation of China under Grant 61503033. Y. Tang is with the School of Automation, Beijing University of Posts and Telecommunications, Beijing 100876, China (e-mail: yttang@bupt.edu.cn).

Abstract: In this paper, we study a distributed optimization problem for a class of high-order multi-agent systems with unknown dynamics. In comparison with existing results for integrators or linear agents, we need to overcome the difficulties brought by the unknown nonlinearities and also the optimization requirement. For this purpose, we employ an embedded control based design and first convert this problem into an output stabilization problem. Then, two kinds of adaptive controllers are given for these agents to drive their outputs to the global optimal solution under some mild conditions. Finally, we show that the estimated parameter vector converges to the true parameter vector under some well-known persistence of excitation condition. The efficacy of these algorithms was verified by a simulation example.

Keywords: Distributed optimization, unknown dynamics, embedded design, adaptive control, parameter convergence.

1 Introduction

Over the past few years, distributed optimization of multi-agent systems has become a hot topic due to the fast development of multi-robot networks, machine learning and big data technologies [1, 2, 3]. In a typical setting of this problem, each agent is assigned with a local cost function and the control objective is to propose distributed controls that guarantee a consensus on the optimal solution of the sum of all local cost functions. Many effective algorithms have been proposed to achieve this goal in different situations [4, 5, 6, 7, 8, 9].

Here, we follow this technical line but consider high-order continuous-time nonlinear agents unknown dynamics. While most of the existing works were only devoted to single-integrator agents, there are many distributed optimization tasks implemented by or depending on physical plants of continuous dynamics in practice, e.g. source seeking in multi-robot systems [10], attitude formation control of rigid bodies[11], optimal power dispatch over power networks [12]. Note that these physical dynamics can hardly be described well by single integrators. In fact, an example was given to show that the algorithms designed for single integrators might fail to achieve a desired performance for these high-order agents [13]. Thus, we have to take the high-order and possible nonlinear agents’ dynamics into account to achieve the distributed optimization goal.

In light of the optimization requirement for agents, the gradient-based closed-loop systems are basically nonlinear. Furthermore, the high-order feature of these agents brings many new difficulties to the associated analysis and design, which makes this problem much more challenging. In fact, very few optimization results have been obtained on this topic with continuous-time agents in the form of high-order dynamics. For example, a distributed algorithm for double integrators was proposed in [14] with an integral control idea and then extended to Euler-Lagrange agents. A class of nonlinear minimum phase agents in output feedback form with unity relative degree was considered by an internal-model design [15]. Distributed optimization problem with bounded controls was also explored for both single and double integrators [16] . However, distributed optimization of general high-order multi-agent systems is still far from being solved.

Recently, we proposed an embedded control scheme to solve this problem for general linear systems having well-defined vector relative degrees [17]. To overcome the difficulties brought by both high-order dynamics and possible nonlinearities from gradients, we divided the design into two main parts: optimal signal generator construction and reference tracker design. This embedded technique makes the control design carried out in a “separative” way, so as to simplify the whole design by almost independently tackling the optimal consensus problem for single integrators and output tracking problem of high-order agents.

Note that exact information of system matrices is required in this embedded design, which may be not available or with measurement errors in applications. This motivates us to investigate the distributed optimization problem for high-order nonlinear agents with unknown time-varying dynamics. Furthermore, we focus on the case when this unknown time-varying nonlinearity can be linearly parameterized. The objective of this paper is to extend existing embedded control results to this class of uncertain high-order nonlinear agents and achieve the distributed optimization goal.

In view of the aforementioned observations, the contribution of this paper is at least two-fold. Firstly, a distributed optimization problem was formulated and solved for a group of high-order nonlinear agents, which can be taken as an extended version of existing results for single integrators [2, 7]. Secondly, the embedded control technique proposed in [17] was further explored and extended to solve the distributed optimization problem of high-order multi-agent systems with unknown dynamics. By removing the requirement of knowing agents’ exact dynamics, this work can be taken as an adaptive extension to existing results for linear multi-agent systems [7, 16, 17] and nonlinear ones with unity relative degree [15]. Moreover, the obtained conclusions can be applied to achieve an output average consensus for these uncertain nonlinear agents, while only integrators or linear agents were considered in existing works [18, 19, 20].

The remainder of this paper is organized as follows. Problem formulation is presented in Section 2. Then the main result is presented in Sections 3 along with both stability analysis and parameter convergence. Following that, an example is given to illustrate the effectiveness of our algorithms in Section 4. Finally, concluding remarks are given in Section 5.

Notations: Let $\mathbb{R}^{n}$ be the $n$-dimensional Euclidean space. For a vector $x$, $||x||$ denotes its Euclidean norm. ${\bf 1}_{N}$ (and ${\bf 0}_{N}$) denotes an $N$-dimensional all-one (and all-zero) column vector. $\mbox{diag}\{b_{1},\,{\dots},\,b_{n}\}$ denotes an $n\times n$ diagonal matrix with diagonal elements $b_{i},\,(i=1,\,{\dots},\,n)$. $\mbox{col}(a_{1},\,{\dots},\,a_{n})=[a_{1}^{\mathrm{T}},\,{\dots},\,a_{n}^{\mathrm{T}}]^{\mathrm{T}}$ for column vectors $a_{i}\;(i=1,\,{\dots},\,n)$. A weighted directed graph (or digraph) $\mathcal{G}=(\mathcal{N},\mathcal{E},\mathcal{A})$ is defined as follows, where $\mathcal{N}=\{1,{\dots},n\}$ is the set of nodes, $\mathcal{E}\subset\mathcal{N}\times\mathcal{N}$ is the set of edges, and $\mathcal{A}\in\mathbb{R}^{n\times n}$ is a weighted adjacency matrix [21]. $(i,j)\in\mathcal{E}$ denotes an edge leaving from node $i$ and entering node $j$. The weighted adjacency matrix of this digraph $\mathcal{G}$ is described by $A=[a_{ij}]\in\mathbb{R}^{n\times n}$, where $a_{ii}=0$ and $a_{ij}\geq 0$ ($a_{ij}>0$ if and only if there is an edge from agent $j$ to agent $i$). A path in graph $\mathcal{G}$ is an alternating sequence $i_{1}e_{1}i_{2}e_{2}{\cdots}e_{k-1}i_{k}$ of nodes $i_{l}$ and edges $e_{m}=(i_{m},\,i_{m+1})\in\mathcal{E}$ for $l=1,\,2,\,{\dots},\,k$. If there exists a path from node $i$ to node $j$ then node $i$ is said to be reachable from node $j$. The neighbor set of agent $i$ is defined as $\mathcal{N}_{i}=\{j\colon(j,\,i)\in\mathcal{E}\}$ for $i=1,\,\dots,\,n$. A graph is said to be undirected if $a_{ij}=a_{ji}$ ($i,\,j=1,\,{\dots},\,n$). An undirected graph is said to be connected if there is a path between any two vertices. The weighted Laplacian $L=[l_{ij}]\in\mathbb{R}^{n\times n}$ of graph $\mathcal{G}$ is defined as $l_{ii}=\sum\nolimits_{j\neq i}a_{ij}$ and $l_{ij}=-a_{ij}(j\neq i)$.

2 Problem formulation

In this paper, we consider a collection of heterogeneous high-order nonlinear systems described by:

[TABLE]

where $x_{j,i}\in\mathbb{R}$ is the $j$-th state variable of agent $i$, $x_{i}\triangleq\mbox{col}(x_{1,i},\,\dots,\,x_{n_{i},i})\in\mathbb{R}^{n_{i}}$, $y_{i}\in\mathbb{R}$ and $u_{i}\in\mathbb{R}$ are respectively the state, output, and input of agent $i$. The function $\Delta_{i}(x_{i},\,\theta_{i},\,t)$ represents the unknown time-varying nonlinearities with uncertain parameter $\theta_{i}=\mbox{col}(\theta_{1,i},\,\dots,\,\theta_{n_{\theta_{n_{i}}},i})\in\mathbb{R}^{n_{\theta_{i}}}$, which might result from modeling errors or external perturbations.

Associated with this multi-agent system, each agent is endowed with a differentiable local cost function $f_{i}\colon\mathbb{R}\to\mathbb{R}$. The global cost function is defined as the sum of local costs, i.e., $f(y)=\sum\nolimits_{i=1}^{N}f_{i}(y)$. Moreover, we assume the local cost function $f_{i}(\cdot)$ is only known to agent $i$ itself and cannot be shared globally in the multi-agent network. Coupled with these nonlinear agents, we aim to design proper controllers such that the outputs of these agents asymptotically minimize the global cost function.

To clarify our following design, we focus on a class of systems in the form of (1) as follows.

Assumption 1**.**

For any $i=1,\,\dots,\,N$, there exist a known basis function vector ${\bm{p}}_{i}(x_{i},\,t)$ and an unknown parameter vector $\theta_{i}\in\mathbb{R}^{n_{\theta_{i}}}$ satisfying $\Delta_{i}(x_{i},\,\theta_{i},\,t)=\theta_{i}^{\mathrm{T}}{\bm{p}}_{i}(x_{i},\,t)$ for all $x_{i}\in\mathbb{R}^{n_{i}}$, $t\geq 0$ and $\theta_{i}\in\mathbb{R}^{n_{\theta_{i}}}$. Furthermore, ${\bm{p}}_{i}(x_{i},\,t)$ can be uniformly bounded by smooth functions of $x_{i}$.

Remark 1**.**

The unknown nonlinear dynamics of all agents are assumed to be linearly parameterized, which have been widely studied in both classical adaptive control and multi-agent coordination literature [22, 23, 24, 25]. In fact, a plenty of practical systems can be put into this form, which is general enough to cover integrators, Van der Pol systems, Duffing equations and many mechanical systems. Moreover, this time-varying feature can further be used to represent many typical external disturbances, e.g. constants and sinusoidal signals.

The following assumption is often made in convex optimization literature [26, 7, 27], which guarantees the existence and uniqueness of the optimal solution to problem (2).

Assumption 2**.**

For $i=1,\,\dots,\,N$, the function $f_{i}(\cdot)$ is $\underline{l}$-strongly convex and its gradient is $\overline{l}$-Lipschitz for constants $\underline{l},\,\overline{l}>0$.

As usual, we assume this optimal solution is finite and denote it as $y^{*}$, i.e.

[TABLE]

Due to the privacy of local cost function $f_{i}(\cdot)$, no agent can gather enough information to determine the global optimal solution $y^{*}$ by itself. Hence, our problem cannot be solved without cooperation and information sharing among these agents.

For this purpose, we use a weighted undirected graph $\mathcal{G}=(\mathcal{N},\mathcal{E},\mathcal{A})$ to describe the information sharing topology with note set $\mathcal{N}=\{1,\,{\dots},\,N\}$. An edge $(i,\,j)\in\mathcal{E}$ between nodes $i$ and $j$ means that agent $i$ and agent $j$ can share information with each other. Suppose the following assumption is fulfilled as in many publications [7, 28, 14, 17, 9].

Assumption 3**.**

The graph $\mathcal{G}$ is connected.

This assumption is about the connectivity of information sharing graph $\mathcal{G}$, which guarantees that any agent’s information can reach any other agents. Under this assumption, it is well-known that the associated Laplacian $L$ is positive semidefinite with null space spanned by ${\bm{1}}_{N}$ [21].

The distributed optimization problem considered in this paper is readily formulated as follows.

Problem 1**.**

For given agents of the form (1), information sharing graph $\mathcal{G}$ and local cost function $f_{i}(\cdot)$, if possible, determine a distributed protocol for each agent by using its own local data and exchanged information with its neighbors such that

all the trajectories of agents are bounded over the time interval $[0,\,+\infty)$.

• 2)

their outputs of agents satisfy

[TABLE]

Remark 2**.**

The formulated problem can be taken as a combination of the well-studied topics: distributed optimization [2, 3, 7, 29] and output consensus [18, 30, 31]. Since an output consensus of the whole high-order multi-agent system must be achieved as the solution of a convex optimization problem in the formulated problem, it is certainly more challenging than the existing output consensus results for high-order agents.

Remark 3**.**

In contrast with existing distributed optimization works, the agents considered here are high-order and heterogeneous, while only single integrators are considered in many publications [4, 8, 32, 27]. Moreover, we study the case when these agents are with unknown dynamics, while the exact information of agents’ dynamics is required in [14] and [17].

Particularly, when the local cost function is chosen as $f_{i}(s)=(s-y_{i}(0))^{2}$, this formulation can solve an output average consensus problem for these high-order uncertain nonlinear agents and thus includes exiting results for integrators and linear systems as special cases [18, 19, 20].

As mentioned above, the main difficulty to solve the distributed optimization for these agents lies in the coupling of the high-order structure associated with agents’ dynamics and the global optimization requirement. To overcome this point, we adopt the embedded control scheme [17] and propose adaptive control laws to solve the distributed optimization problem for agent (1) in the following section.

3 Main Results

In this section, we first employ an embedded control approach to convert our problem into an output stabilization problem, and then propose distributed adaptive algorithms for these high-order nonlinear agents with unknown dynamics to achieve the optimization goal along with parameter convergence analysis.

3.1 Embedded Control Design

The embedded control approach was first proposed by Tang et al [17] to solve the distributed optimization problem for high-order linear agents. In this approach, an optimal signal generator must be constructed first by considering the same optimization problem (2) for a group of single-integrator agents, in order to asymptotically reproduce the optimal solution $y^{*}$ by a signal $r_{i}$. Then, by taking $r_{i}$ as an output reference signal for agent $i$, this generator is embedded in the feedback loop via some proper interfaces for original agents. In this way, the distributed optimization problem for general linear agents is divided into two simpler subproblems, i.e., construction of an optimal signal generator for single integrators and design of proper trackers for linear agents with output references, which can be independently solved in a modular way.

The first subproblem is essentially a conventional distributed optimization problem for single integrators and has been well-studied in existing literature [7, 8, 29]. To solve this problem, the following optimal signal generator was proposed for problem (2) in [17].

[TABLE]

The effectiveness of (4) was already proven in [17]. We repeat it as follows for completeness.

Lemma 1**.**

Suppose Assumptions 2 and 3 hold. Then, along the trajectory of system (4), $r_{i}(t)$ approaches the optimal solution $y^{*}$ exponentially as $t\to\infty$ for $i=1,\,\ldots,\,N$.

Suppose we know the analytical form of $f_{i}(\cdot)$ or at least $\nabla f_{i}(\cdot)$, the optimal signal generator can be implemented independently to generate the minimizer of problem (2) as showed in this lemma. However, this requirement may cost too much in applications and even impossible in many practical circumstances. Thus, we follow an oracle-based description[33] of $f_{i}(\cdot)$ and consider the case when only the real-time gradient $\nabla f_{i}(y_{i})$ is available upon requests in our following design.

Furthermore, exact information of system matrices is required for the proposed controllers in [17]. This implies that the corresponding designs can not be directly used to solve our problem for uncertain agents in the form of (1). Thus, the distributed optimization problem considered in this paper is much more challenging than those addressed for integrators or linear agents in existing works [7, 4, 27, 8, 17].

To deal with these two issues, we adopt a certainty-equivalence design and propose an algorithm as follows:

[TABLE]

where $\hat{\theta}_{i}$ is the estimation of uncertain vector $\theta_{i}$, and the constants $k_{1i},\,\dots,\,k_{n_{i}i}$ and function $\phi_{i}(\cdot)$ to be specified later. Here the constant $\varepsilon>0$ is a tunable high-gain parameter to deal with the real-time gradient issue.

Remark 4**.**

Here, the variables $\hat{\theta}_{i},\,r_{i},\,\lambda_{i}$ constitute the local compensator of agent $i$. It can be verified that this control is indeed distributed in the sense of only using the agents’ own local data and exchanged information with their neighbors. Moreover, from its nominal form (5), the above control is composed of two parts, where the last two subsystems can be taken as a modified version of optimal signal generator (4) with real-time gradients and the rest compose an adaptive tracking controller such that $y_{i}(t)$ can track $r_{i}(t)$ as $t$ goes to infinity in spite of those unknown dynamics, which confirms the embedded design methodology.

Under the above control law, the composite system is then:

[TABLE]

By letting $\hat{x}_{i}=\mbox{col}(x_{1,i}-r_{i},\,\varepsilon x_{2,i},\,\dots,\,\varepsilon^{n_{i}-1}x_{n_{i},i})$, it can be further rewritten as:

[TABLE]

where $b_{1i}=\mbox{col}(1,\,0,\,\dots,\,0)$, $b_{2i}=\mbox{col}(0,\,\dots,\,0,\,1)$ and A_{i}=\left[\begin{array}[]{c|c}0&I_{n_{x_{i}}-1}\\ \hline\cr k_{1i}&[k_{2i}\,\dots\,k_{n_{i}\,i}]\end{array}\right].

Note that the above system is almost in a singularly perturbed form except the adaption dynamics $\hat{\theta}_{i}$ [34]. By letting $\varepsilon=0$, we have $\hat{x}_{i}=0$ and $x_{1,i}=r_{i}$. The resultant quasi-state-state model of the above composite system is exactly the optimal signal generator (4), which in turn guarantees $x_{1,\,i}(t)\to y^{*}$ as $t$ goes to infinity by Lemma 1.

Based on these observations, our formulated distributed optimization problem for agent (1) is converted to a decentralized output stabilization problem for the above system (7) with output $\hat{x}_{i}$. In other words, we need to determine proper function $\phi_{i}(\cdot)$ and constant $\varepsilon>0$ such that all trajectories of (7) is bounded over $[0,\,+\infty)$ and satisfying $\hat{x}_{i}(t)\to 0$ as $t$ goes to infinity.

3.2 Solvability Analysis

To solve our problem, we denote $\hat{x}=\mbox{col}(\hat{x}_{1},\,\dots,\,\hat{x}_{N})$,  $\theta=\mbox{col}(\theta_{1},\,\dots,\,\theta_{N})$,  $\hat{\theta}=\mbox{col}(\hat{\theta}_{1},\,\dots,\,\hat{\theta}_{N})$,  $r=\mbox{col}(r_{1},\,\dots,\,r_{N})$,  $\lambda=\mbox{col}(\lambda_{1},\,\dots,\,\lambda_{N})$ and $\bar{\theta}=\theta-\hat{\theta}$. The whole composite system can be put into a compact form as follows.

[TABLE]

where $\tilde{f}(y)=\sum\nolimits_{i=1}^{N}f_{i}(y_{i})$,  $A=\mbox{blockdiag}(A_{1},\,\dots,\,A_{N})$,  $B_{1}=\mbox{blockdiag}(b_{11},\,\dots,\,b_{1N})$, $B_{2}=\mbox{blockdiag}(b_{21},\,\dots,\,b_{2N})$, $E=\mbox{blockdiag}(\varepsilon^{n_{1}-1}I_{n_{1}},\,\dots,\,\varepsilon^{n_{N}-1}I_{n_{N}})$,  ${\bm{p}}(x,\,t)\triangleq\mbox{blockdiag}({\bm{p}}_{1}(x_{1},\,t),\,\dots,\,{\bm{p}}_{N}(x_{N},\,t))$,  $L$ is the Laplacian of $\mathcal{G}$, and the function $\phi(x,\,\hat{\theta},\,r,\,t)$ is determined by $\phi_{1}(\cdot),\,\dots,\,\phi_{N}(\cdot)$.

We first choose gain constants $k_{1i},\,\dots,\,k_{n_{i}\,i}$ such that the polynomial $s^{n_{i}}-k_{n_{i}\,i}s^{n_{i}-1}-k_{2i}s-k_{1i}$ is Hurwitz for any $1\leq i\leq N$. This implies that the following Lyapunov equation

[TABLE]

has a unique positive definite solution $P_{i}$ with compatible dimensions.

Inspired by existing Lyapunov-based designs [22, 24], we present the main result of this paper as follows.

Theorem 1**.**

Suppose Assumptions 1–3 hold. Then, the distributed optimization problem determined by (1) and (2) can be solved by controllers of the form (5) with $\phi_{i}(x_{i},\,\hat{\theta}_{i},\,r_{i},\,t)={\bm{p}}_{i}(x_{i},\,t)b_{2i}^{\mathrm{T}}P_{i}\hat{x}_{i}$ for a small enough $\varepsilon>0$, where the constant $k_{ji}$ is chosen as above for $i=1,\,\dots,\,N,\,j=1,\,\dots,\,n_{i}$.

**Proof. ** The proof is mainly based on system composition techniques.

Step 1: consider the first two subsystems of (8). Let $\hat{V}_{i}=\hat{W}_{i}+{\varepsilon^{n_{i}-1}}\bar{\theta}_{i}^{\mathrm{T}}\bar{\theta}_{i}$ with $\hat{W}_{i}=\hat{x}_{i}^{\mathrm{T}}P_{i}\hat{x}_{i}$. Its time derivative along the trajectory of the composite system (7) satisfies

[TABLE]

By Young’s inequality, it holds that

[TABLE]

for some known positive constants $c_{1}$ and $c_{2}$.

Choose $\hat{V}=\sum\nolimits_{i=1}^{N}\hat{V}_{i}$. We further have

[TABLE]

Step 2: consider the last two subsystems of (8), which can be rewritten as

[TABLE]

where the function ${\bm{h}}(r,\,y)\triangleq\nabla\tilde{f}(r)-\nabla\tilde{f}(y)$ is $\overline{l}$-Lipschitz in $r-y$ by Assumption 2.

By taking ${\bm{h}}(r,\,y)$ as perturbations, we let $\mbox{col}(r^{*},\lambda^{*})$ be the equilibrium point of unperturbed $(r,\,\lambda)$-system (i.e. when ${\bm{h}}(r,\,y)\equiv{\bm{0}}$). Note that the unperturbed system is exactly the optimal signal generator, it implies $r^{*}={\bm{1}}_{N}y^{*}$ under Assumptions 2 and 3 by Lemma 1. In fact, the equilibrium point satisfies $-\nabla f(r^{*})-L\lambda^{*}={\bm{0}}_{N}$ and $Lr^{*}={\bm{0}}_{N}$. As a result, there exists a constant $\theta$ such that $r_{1}^{*}=\dots=r_{N}^{*}=\theta$ since the null space of $L$ is spanned by ${\bm{1}}_{N}$ under Assumption 3. Then, one can obtain $\sum\nolimits_{i=1}^{N}\nabla f_{i}(\theta)=0$ by ${\bm{1}}_{N}^{\mathrm{T}}L={\bm{0}}_{N}$, which implies that $\theta$ is an optimal solution of (2). By Assumption 2, this implies $\theta=y^{*}$ and $r^{*}={\bm{1}}_{N}y^{*}$.

Letting $\bar{r}=r-r^{*}$ and $\bar{\lambda}=\lambda-\lambda^{*}$ gives

[TABLE]

Inspired by the proof of Lemma 1 in [17] , we let $R\in\mathbb{R}^{N\times(N-1)}$ be a matrix satisfying $R^{\mathrm{T}}{\bm{1}}_{N}={\bm{0}}_{N}$, $R^{\mathrm{T}}R=I_{N-1}$, and $RR^{\mathrm{T}}=I_{N}-\frac{1}{N}{\bm{1}}_{N}{\bm{1}}_{N}^{\mathrm{T}}$. Apparently, the matrix $R$ has a full column rank. Denote $T=[\frac{{\bm{1}}_{N}}{\sqrt{N}}~{}R]^{\mathrm{T}}$ and perform a coordinate transform $\hat{\lambda}=T\bar{\lambda}$. The above system is equivalent to $\dot{\hat{\lambda}}_{1}=0$ and

[TABLE]

Furthermore, the unperturbed system is globally exponential stable at the origin under Assumptions 2 and 3 by Lemma 1.

To apply the system composition arguments, one has to investigate the robustness of system (11). For this purpose, we let $\hat{r}\triangleq\mbox{col}(\bar{r},\,\hat{\lambda}_{2})$ and apply the converse Lyapunov theorem (Theorem 4.15 in [34]) to the unperturbed subsystem, that is, there is a continuously differentiable Lyapunov function $\bar{V}(\cdot)$ such that

[TABLE]

for some positive constants $c_{3},\,\dots,\,c_{6}$.

Along the trajectory of perturbed system (11), one can obtain:

[TABLE]

where we use the Lipschitzness of ${\bm{h}}(r,\,y)$ in $r-y$ and thus in $\hat{x}$.

Step 3: consider the stability of the composite system composed by the first two subsystems of (8) and system (11). Let $V=\hat{V}+c\bar{V}$ with $c>0$ to be specified later. By using equalities (10) and (12), the derivative of $V$ with respect to $t$ along the whole composite system satisfies

[TABLE]

Note that $-\nabla\tilde{f}(r)-L\lambda$ and ${\bm{h}}(r,\,y)$ are both globally Lipschitz in their arguments $\hat{r}$ and $\hat{x}$ under Assumption 2. From (11), one can determine a known constant $c_{7}>0$ satisfying that $||\dot{r}||^{2}=||\dot{\bar{r}}||^{2}\leq c_{7}(||\hat{r}||^{2}+||\hat{x}||^{2})$

By Young’s inequality, we have

[TABLE]

Letting $c>\frac{c_{2}c_{7}+c_{6}^{2}\bar{l}^{2}+1}{c_{5}}$ and $\varepsilon<\frac{c_{1}}{c_{2}c_{7}+c^{2}+1}$ gives that

[TABLE]

According to the LaSalle-Yoshizawa theorem (Theorem 2.1 in [22]), we have that all trajectories of the closed-loop system composed of (8) and (11) are bounded over the time interval $[0,\,+\infty)$ and satisfy that $\lim_{t\to\infty}||\hat{x}_{i}(t)||+||\hat{r}(t)||=0$. As immediate results, one can conclude the boundedness of $x(t)$, $r(t)$, $\lambda(t)$ and $\hat{\theta}(t)$. Moreover, we can obtain that $\lim_{t\to\infty}\hat{x}_{1,i}(t)=0$ and $\lim_{t\to\infty}r_{i}(t)=y^{*}$, which implies that

[TABLE]

as $t\to+\infty$. The proof is thus complete.

Remark 5**.**

Note that we consider uncertain high-order agents in the form of (1), which includes integrators as its special cases. Thus, the theorems can be taken as adaptive extensions of existing results with exact known dynamics[14, 19, 20, 17]. Moreover, many actuating disturbances can be represented in the form of (1), thus we provide different methods to achieve disturbance rejection from the internal model-based approach used in existing works [28, 15].

Particularly, when the analytical form of $f_{i}(\cdot)$ or $\nabla f_{i}(\cdot)$ is known to us, the optimal signal generator can be implemented independently. Following a similar proof, we can choose the gain parameter $\varepsilon$ as any positive constant to solve our problem.

Theorem 2**.**

Suppose Assumptions 1–3 hold. Then, the distributed optimization problem determined by (1) and (2) can be solved by the following control

[TABLE]

where the constant $k_{ji}$ is chosen as above and $\varepsilon>0$ is arbitrary for $i=1,\,\dots,\,N,\,j=1,\,\dots,\,n_{i}$.

Remark 6**.**

In some circumstances, we may further let $\phi_{i}(x_{i},\,\hat{\theta}_{i},\,r_{i},\,t)=\Lambda_{i}{\bm{p}}_{i}(x_{i},\,t)b_{2i}^{\mathrm{T}}P_{i}\hat{x}_{i}$ with a positive definite matrix $\Lambda_{i}$. This matrix is called an adaption gain in literature[35]. It can be used to achieve a fast adaption and then improve the transient performance of our controllers to solve the distributed optimization problem.

Remark 7**.**

Without further information of the unknown dynamics, the two controllers may fail in practical applications if there are external disturbances or noises in measurements of $x_{i}$, although it is theoretically proved to achieve the optimization goal as $t$ goes to infinity. To tackle this problem, we can employ a $\sigma$-modification [35] for $\hat{\theta}_{i}$ with sacrificing some accuracy in control performance as follows:

[TABLE]

where $\sigma_{\theta_{i}}>0$ is a tunable parameter such that $\lim_{t\to+\infty}||y_{i}(t)-y^{*}||$ can be smaller than any desired positive constant.

3.3 Parameter Convergence

It has been shown that parameter convergence is essential in achieving robustness of the proposed adaptive controllers[35, 22, 36]. From the proof of Theorem 1, one can merely conclude that the estimator $\hat{\theta}_{i}$ converges to some constant. In this subsection, we assert conditions under which $\hat{\theta}_{i}(t)$ will converge to its true value $\theta_{i}$ as $t$ tends to infinity.

For this purpose, we further assume the basis function ${\bm{p}}_{i}(x_{i},\,t)$ satisfying the following condition.

Assumption 4**.**

For any $i=1,\,\dots,\,N$, along the trajectory of the closed-loop system composed of (1) and (5) or (13), there exist positive constants $m$, $t_{0}$, $T_{0}$ such that the function ${{\bm{p}}}_{i}(x_{i}(t),\,t)$ is uniformly bounded and the following inequality is satisfied:

[TABLE]

Note that $x_{i}(t)$ is ultimately bounded by Theorem 1, the boundedness of ${\bm{p}}_{i}(x_{i}(t),\,t)$ is not too strict. The inequality (15) is often known as the persistence of excitation (PE) condition, which has widely used in adaptive control literature [22, 24].

Theorem 3**.**

Suppose Assumptions 1–4 hold. Then, along the trajectory of system (1) under the controllers proposed in Theorems 1 and 2 , it holds that $\lim_{t\to+\infty}\hat{\theta}_{i}(t)=\theta_{i}$ for $i=1,\,\dots,\,N$.

**Proof. ** To show this theorem, we first claim that $\lim_{t\to+\infty}\bar{\theta}_{i}^{\mathrm{T}}(t){\bm{p}}_{i}(x_{i}(t),\,t)=0$. By the proof of Theorem 1, we have $\hat{x}_{i}(\infty)=\int_{0}^{+\infty}\dot{\hat{x}}_{i}(\tau){\mathrm{d}}\tau=0$. From the uniform boundedness of associated variables and Assumption 4, it follows that $\ddot{\hat{x}}_{i}(t)$ is also bounded. Using Barbalat’s lemma (Lemma 8.2 in [34]) to $\dot{\hat{x}}_{i}(t)$ implies that $\lim_{t\to+\infty}\dot{\hat{x}}_{i}(t)=0$, which confirms this claim.

Next, noting $\dot{\bar{\theta}}_{i}=\dot{\hat{\theta}}_{i}={\bm{p}}_{i}(x_{i},\,t)b_{2i}^{\mathrm{T}}P_{i}\hat{x}_{i}$ gives $\lim_{t\to+\infty}\dot{\bar{\theta}}_{i}(t)=0$ by Assumption 1. According to Lemma 1 in [37] or its proof, the two facts $\lim_{t\to+\infty}\dot{\bar{\theta}}_{i}(t)=0$ and $\lim_{t\to+\infty}\bar{\theta}_{i}^{\mathrm{T}}(t){\bm{p}}_{i}(x_{i}(t),\,t)=0$ provide us that $\lim_{t\to+\infty}\bar{\theta}_{i}(t)=0$ under Assumption 4. The proof is thus complete.

Remark 8**.**

Since the unknown dynamics is linearly parameterized by Assumption 1, this theorem can be further modified and applied to any number of components in ${\bm{p}}_{i}(x_{i},\,t)$ satisfying such a PE condition, and then used to address the parameter convergence problem in a more precise way. Specially, when the basis function is time-invariant, the $j$-th component ${\bm{p}}_{j,i}(x_{i})$ of ${\bm{p}}_{i}(x_{i})$ is persistently excited if $\lim_{x_{i}\to\mbox{col}(y^{*},\,0,\,\dots,\,0)}{\bm{p}}_{j,i}(x_{i})\neq 0$ and guarantees the convergence of $\hat{\theta}_{j,i}(t)$ to $\theta_{j,i}$ as $t$ goes to infinity.

4 Simulations

In this section, we present a numerical example to illustrate our problem and the effectiveness of our designs.

Consider a multi-agent system including four controlled Van der Pol systems as follows.

[TABLE]

where $\Xi_{i}\triangleq-a_{i}x_{1,i}+b_{i}(1-x_{1,i}^{2})x_{2,i}$ with $a_{i},\,b_{i}>0$ but unknown. The trajectories of the unforced system with different initial conditions when $a_{i}=b_{i}=1$ are depicted in Figure 1.

To make it more interesting, we further assume that agent $i$ is subject to an actuating disturbance $d_{i}(t)$ described by

[TABLE]

where $D_{i}=[1~{}0]$ and $S_{i}=[0~{}1;\,-1~{}0]$.

The information sharing graph of this multi-agent system is depicted in Figure 2 with unity edge weights. The local cost functions are as follows.

[TABLE]

Denote $\Delta_{i}(x_{i},\,t)=\Xi_{i}+d_{i}(t)$. The agents are of the form (1). Note that $d_{i}(t)=A_{1i}\sin(t)+A_{2i}\cos(t)$ for some constants $A_{1i}$,  $A_{2i}$ depending upon $v_{i}(0)$. By letting $\theta_{i}=\mbox{col}(\theta_{1,i},\,\dots,\,\theta_{4,i})=\mbox{col}(a_{i},\,b_{i},\,A_{1i},\,A_{2i})$ and ${\bm{p}}_{i}(x_{i},\,t)=\mbox{col}(-x_{1,i},\,(1-x_{1,i}^{2})x_{2,i},\,\sin(t),\,\cos(t))$, one can find that Assumption 1 is fulfilled. Moreover, both Assumptions 2 and 3 are also verified. Additionally, the optimal solution is $y^{*}\approx 3.24$ by numerically minimizing the global cost function $f(y)=\sum\nolimits_{i=1}^{4}f_{i}(y)$. According to Theorem 1, the distributed optimization problem for these agents can be solved by a controller of the form (5).

For simulations, we choose the parameters as $a_{i}=b_{i}=1$, $v_{i}(0)=\mbox{col}(1,\,0)$, $k_{1i}=-4$, $k_{2i}=-4$, $\Lambda_{i}=10I_{4}$ and $\varepsilon=0.2$. The profiles of agents’ outputs are shown in Figure 3. Satisfactory performance is observed. For parameter convergence, we have ${\bm{p}}_{1,i}({\bm{x}}_{i},\,t)=-x_{1,i}$, which is time-invariant and satisfies that $\lim_{x_{i}\to\mbox{col}(y^{*},\,0,\,\dots,\,0)}{\bm{p}}_{1,i}({\bm{x}}_{i})=-y^{*}\neq 0$. By some calculations, one can also determine that

[TABLE]

Using Theorem 3, we can conclude that the estimators $\hat{\theta}_{1,i}$, $\hat{\theta}_{3,i}$,  $\hat{\theta}_{4,i}$ will converge to their true values, while $\hat{\theta}_{2,i}$ may fail. This conclusion is confirmed by Figures 4 and 5.

5 Conclusions

A distributed optimization problem was formulated for a class of high-order nonlinear systems with unknown dynamics. By using an embedded control scheme, we proposed distributed adaptive controls to solve this problem under standard assumptions and parameter convergence was also addressed. Output feedback control with directed information sharing graphs will be our future work.