Distributed Optimal Steady-State Regulation for High-Order Multi-Agent Systems with External Disturbances

TL;DR

This paper develops a distributed control approach for high-order multi-agent systems to achieve optimal steady-state regulation despite external disturbances, using embedded control design and high-gain techniques.

Contribution

It introduces a novel distributed control framework that handles heterogeneity, disturbances, and real-time gradient information for multi-agent systems.

Findings

Effective disturbance rejection demonstrated through simulations

Distributed algorithms achieve optimal steady-state regulation

Extension to output feedback with high-gain control

Abstract

In this paper, a distributed optimal steady-state regulation problem is formulated and investigated for heterogeneous linear multi-agent systems subject to external disturbances. We aim to steer this high-order multi-agent network to a prescribed steady-state determined as the optimal solution of a resource allocation problem in a distributed way. To solve this problem, we employ an embedded control design and convert the formulated problem to two simpler subproblems. Then, both state-feedback and output feedback controls are presented under mild assumptions to solve this problem with disturbance rejection. Moreover, we extend these results to the case with only real-time gradient information by high-gain control techniques. Finally, numerical simulations verify their effectiveness.

Equations

$$\begin{array}{rl}{\dot{x}}_i& =A_i{x}_i+B_i{u}_i+E_i{\omega }_i\\ y_i& =C_i{x}_i,i=1,\dots ,N\end{array}$$

$${\dot{\omega }}_i=S_i{\omega }_i.$$

$$\begin{array}{rl}\text{mbox}minimize& f(y)={\sum }_{i=1}^N{f}_i(y_i)\\ \text{mbox}subjectto& {\sum }_{i=1}^N{y}_i={\sum }_{i=1}^N{d}_i\end{array}$$

$${\underline{h}}_i\le {\nabla }^2{f}_i(s)\le {\bar{h}}_i,\forall s\in \mathbb{R}.$$

$$\begin{array}{rl}{X}_{i1}{S}_i& =A_i{X}_{i1}+B_i{U}_{i1}+E_i\\ 0& =C_i{X}_{i1}\end{array}$$

$$\begin{array}{rl}0& =A_i{X}_{i2}+B_i{U}_{i2}\\ 1& =C_i{X}_{i2}.\end{array}$$

$$\mathrm{rank}\left[\begin{array}{cc}{A}_i-\lambda I_{n_i}& B_i\\ C_i& 0\end{array}\right]=n_i+1,\forall \lambda \in \sigma (S_i)\cup 0,i=1,\dots ,N.$$

$$\begin{array}{rl}{\dot{z}}_i& =-\nabla f_i(z_i)+{\lambda }_i\\ {\dot{\lambda }}_i& =-{\lambda }_i^0-v_i^0+d_i-z_i\\ {\dot{v}}_i& ={\lambda }_i^0\end{array}$$

$$\begin{array}{rl}{\dot{\bar{z}}}_i& =-{\mathbf{h}}_i({\bar{z}}_i,z_i^{\ast })+{\bar{\lambda }}_i\\ {\dot{\bar{\lambda }}}_i& =-{\bar{\lambda }}_i^0-{\bar{v}}_i^0-{\bar{z}}_i\\ {\dot{\bar{v}}}_i& ={\bar{\lambda }}_i^0\end{array}$$

$$\begin{array}{rl}\dot{\bar{z}}& =-\mathbf{h}(\bar{z},z^{\ast })+\bar{\lambda }\\ \dot{\bar{\lambda }}& =-L\bar{\lambda }-L\bar{v}-\bar{z}\\ \dot{\bar{v}}& =L\bar{\lambda }\end{array}$$

$$\begin{array}{rl}\dot{\bar{z}}& =-\mathbf{h}(\bar{z},z^{\ast })+\bar{\lambda }\\ \dot{\bar{\lambda }}& =-L\bar{\lambda }-LR{\hat{v}}_2-\bar{z}\\ {\dot{\hat{v}}}_2& =R^{\mathrm{T}}{L}^{\mathrm{T}}\bar{\lambda }\end{array}$$

$$\begin{array}{rl}{\dot{\bar{\eta }}}_i& =(S_i-{\bar{L}}_i{E}_i){\bar{\eta }}_i+(S_i{\bar{L}}_i-{\bar{L}}_i{E}_i{\bar{L}}_i-{\bar{L}}_i{A}_i)x_i-{\bar{L}}_i{B}_i{u}_i\\ {\eta }_i& ={\bar{\eta }}_i+{\bar{L}}_i{x}_i\end{array}$$

$${\dot{\hat{\eta }}}_i$$

$$=(S_i-{\bar{L}}_i{E}_i){\bar{\eta }}_i+(S_i{\bar{L}}_i-{\bar{L}}_i{E}_i{\bar{L}}_i-{\bar{L}}_i{A}_i)x_i$$

$$-{\bar{L}}_i{B}_i{u}_i+{\bar{L}}_i(A_i{x}_i+B_i{u}_i+E_i{\omega }_i)-S_i{\omega }_i$$

$$=(S_i-{\bar{L}}_i{E}_i){\hat{\eta }}_i.$$

$$\begin{array}{rl}{u}_i& =K_{i1}{x}_i+K_{i2}{\eta }_i+K_{i3}{z}_i\\ {\dot{\bar{\eta }}}_i& =(S_i-{\bar{L}}_i{E}_i){\bar{\eta }}_i+(S_i{\bar{L}}_i-{\bar{L}}_i{E}_i{\bar{L}}_i-{\bar{L}}_i{A}_i)x_i-{\bar{L}}_i{B}_i{u}_i\\ {\dot{z}}_i& =-\nabla f_i(z_i)+{\lambda }_i\\ {\dot{\lambda }}_i& =-{\lambda }_i^0-v_i^0+d_i-z_i\\ {\dot{v}}_i& ={\lambda }_i^0\end{array}$$

$$\begin{array}{rl}{\dot{\bar{x}}}_i& =(A_i+B_i{K}_{i1}){\bar{x}}_i+B_i{K}_{i2}{\hat{\eta }}_i-X_{i2}{\dot{z}}_i\\ {\dot{\hat{\eta }}}_i& =(S_i-{\bar{L}}_i{E}_i){\hat{\eta }}_i\\ {\dot{z}}_i& =-\nabla f_i(z_i)+{\lambda }_i\\ {\dot{\lambda }}_i& =-{\lambda }_i^0-v_i^0+d_i-z_i\\ {\dot{v}}_i& ={\lambda }_i^0\\ {\bar{y}}_i& =C_i{\bar{x}}_i\end{array}$$

$${\dot{V}}_i^0$$

$$=-2{\bar{x}}_i^{\mathrm{T}}{\bar{x}}_i+2{\bar{x}}_i^{\mathrm{T}}{P}_i{B}_i{K}_{i2}{\hat{\eta }}_i-2{\bar{x}}_i^{\mathrm{T}}{P}_i{X}_{i2}{\dot{z}}_i-c_i{\hat{\eta }}_i^{\mathrm{T}}{\hat{\eta }}_i.$$

$${\dot{V}}_i^0$$

$${\dot{V}}^0$$

$$\le -||\bar{x}|{|}^2-||\hat{\eta }|{|}^2+2{\sum }_{i=1}^N||P_i{X}_{i2}|{|}^2||{\dot{z}}_i|{|}^2.$$

$${\dot{V}}^0\le -||\bar{x}|{|}^2-||\hat{\eta }|{|}^2+c||\dot{z}|{|}^2.$$

$$\begin{array}{rl}\dot{\bar{z}}& =-\mathbf{h}(\bar{z},z^{\ast })+\bar{\lambda }\\ \dot{\bar{\lambda }}& =-L\bar{\lambda }-LR{\hat{v}}_2-\bar{z}\\ {\dot{\hat{v}}}_2& =R^{\mathrm{T}}{L}^{\mathrm{T}}\bar{\lambda }\end{array}$$

$${\hat{c}}_1||\hat{z}|{|}^2\le W(\hat{z})\le {\hat{c}}_2||\hat{z}|{|}^2$$

$$\dot{W}\le -{\hat{c}}_3||\hat{z}|{|}^2,||\frac{\partial W}{\partial \hat{z}}||\le {\hat{c}}_4||\hat{z}||.$$

$$\dot{V}$$

$$\le -||\bar{x}|{|}^2-||\hat{\eta }|{|}^2+\frac{c{\hat{l}}^2}{{\hat{c}}_1}W(\hat{z})-\frac{\hat{c}{\hat{c}}_3}{{\hat{c}}_2}W(\hat{z})$$

$$\le -||\bar{x}|{|}^2-||\hat{\eta }|{|}^2-(\frac{\hat{c}{\hat{c}}_3}{{\hat{c}}_2}-\frac{c{\hat{l}}^2}{{\hat{c}}_1})W(\hat{z})$$

$$\dot{V}$$

$$|y_i-y_i^{\ast }|\le |y_i-z_i|+|z_i-y_i^{\ast }|\to 0\text{ as }t\to \infty .$$

$$\begin{array}{rl}{u}_i& =K_{i1}{\xi }_i+K_{i2}{\eta }_i+K_{i3}{z}_i\\ {\dot{\xi }}_i& =A_i{\xi }_i+B_i{u}_i+{\hat{L}}_i(C_i{\xi }_i-y_i)\\ {\dot{\bar{\eta }}}_i& =(S_i-{\bar{L}}_i{E}_i){\bar{\eta }}_i+(S_i{\bar{L}}_i-{\bar{L}}_i{E}_i{\bar{L}}_i-{\bar{L}}_i{A}_i){\xi }_i-{\bar{L}}_i{B}_i{u}_i\\ {\dot{z}}_i& =-\nabla f_i(z_i)+{\lambda }_i\\ {\dot{\lambda }}_i& =-{\lambda }_i^0-v_i^0+d_i-z_i\\ {\dot{v}}_i& ={\lambda }_i^0\end{array}$$

Full text

Distributed Optimal Steady-State Regulation for High-Order Multi-Agent Systems with External Disturbances

Yutao Tang ***This work was supported by National Natural Science Foundation of China under Grants 61503033. Yutao Tang is with the School of Automation, Beijing University of Posts and Telecommunications, Beijing 100876. P. R. China. Email: yttang@bupt.edu.cn

Abstract: In this paper, a distributed optimal steady-state regulation problem is formulated and investigated for heterogeneous linear multi-agent systems subject to external disturbances. We aim to steer this high-order multi-agent network to a prescribed steady-state determined as the optimal solution of a resource allocation problem in a distributed way. To solve this problem, we employ an embedded control design and convert the formulated problem to two simpler subproblems. Then, both state-feedback and output feedback controls are presented under mild assumptions to solve this problem with disturbance rejection. Moreover, we extend these results to the case with only real-time gradient information by high-gain control techniques. Finally, numerical simulations verify their effectiveness.

Keywords: optimal steady-state regulation, high-order dynamics, embedded control, disturbance rejection

1 Introduction

Multi-agent coordination has been a hot topic over the last decade due to its wide applications in engineering systems. Particularly, steering agents to a prescribed pattern is widely studied in different circumstances. In this type of problems, a (virtual) leader is often set up to describe this pattern or generate references for other agents to follow, while the leader is usually given as a known autonomous dynamic system with possible unknown states. Many results embodying this idea have been obtained in both theoretical and engineering sides (see [1, 2, 3, 4, 5] and references therein).

In this paper, we follow this technical line and consider an optimal steady-state regulation problem for a network of high-order agents. What differentiates our results from existing leader-following literature is that the prescribed pattern of agents here is not given by an autonomous leader but determined as the optimal solution of some resource allocation problem. We further assume that each agent only knows a part of the cost function and resource data. Consequently, the steady-state for this multi-agent system can not be determined off-line without cooperation among these agents due to the distributedness of this optimization problem. In this way, conventional asymptotic regulation techniques fail to solve this problem for a high-order multi-agent system.

When all agents are single integrators, this problem reduces to a special kind of distributed optimization problems, which has been widely investigated from a viewpoint of mathematical programming. In fact, many effective algorithms were proposed [6, 7, 8, 9, 10, 11] to solve this problem under various conditions. Different from these results, the decision variables in this paper are determined as outputs of high-order dynamic systems, which can be deemed as some kind of dynamic constraints. Thus, the solvability of this optimization problem over a high-order multi-agent system by a continuous-time algorithm can be much more challenging than the conventional one. More interestingly, we also consider external disturbances for agents, which are possibly unbounded and disrupt the optimal performance. To guarantee the applicability of designed controllers, it is necessary to consider the disturbance rejection problems in multi-agent control design.

In fact, very few results have been obtained in this topic. Some interesting attempts include [12] and our preceding work [13], where the considered multi-agent systems are subject to an optimal steady-state regulation constraint. While this constraint is defined over the input side for power networks in [12], we formulate and solve the distributed optimal steady-state regulation problem in [13] for a special class of passive systems. Since the relative degree of passive systems is no higher than one, this motivates us to consider more general agents’ dynamics. Although there are similar results to achieve an optimal consensus [14, 15, 16], this is the first time to our knowledge that such a problem is formulated for general linear multi-agent systems subject to external disturbances with a resource-allocation constraint.

In view of the aforementioned observations, the main contributions of this paper are at least two-fold.

• •

A distributed optimal steady-state regulation problem is formulated and solved for a network of high-order agents. Upon the available information, both state-feedback and output-feedback controls are proposed to solve this problem. It can be taken as a distributed version of the conventional asymptotic optimal steady-state optimization and regulation problem for a single plant [17, 18].

• •

Compared with existing distributed resource allocation publications for single integrators, this paper considers general linear multi-agent systems by an embedded control approach and thus includes the single-integrator results as special cases [19, 20].

Furthermore, external disturbances are taken into considerations and rejected by a novel observer-based technique under mild assumptions, while only bounded disturbances can be handled in [15, 13].

The remainder of this paper is organized as follows. In Section 2, we present the formulation part. Then, we convert our problem to two subproblems by embedded controls in Section 3. Main results are presented in Section 4 along with two gradient-based controls. Following that, we extend the above results to the case with only real-time gradients in Section 5 and present several numerical examples to verify the effectiveness of our designs in Section 6. Finally, conclusions are given in Section 7.

Notations: Let $\mathbb{R}^{n}$ be the $n$-dimensional Euclidean space. For a vector $x$, $||x||$ denotes its Euclidean norm. ${\bf 1}_{N}$ (and ${\bf 0}_{N}$) denotes an $N$-dimensional all-one (and all-zero) column vector. $\mbox{col}(a_{1},\,{\dots},\,a_{n})=[a_{1}^{\mathrm{T}},\,{\dots},\,a_{n}^{\mathrm{T}}]^{\mathrm{T}}$ for column vectors $a_{i}\;(i=1,\,{\dots},\,n)$. For a given matrix $A\in\mathbb{R}^{m\times m}$, $\sigma(A)$ denotes its spectrum. For a given smooth function $f\colon\mathbb{R}^{n}\to\mathbb{R}$, we use $\nabla f_{i}$ and $\nabla^{2}f_{i}$ to represent its gradient and Hessian matrix. Let $r_{N}=\frac{1}{\sqrt{N}}{\bf 1}_{N}$ and $R_{N}\in\mathbb{R}^{N\times(N-1)}$ satisfying $R_{N}^{\mathrm{T}}r_{N}={\bf 0}_{N}$, $R_{N}^{\mathrm{T}}R_{N}=I_{N-1}$ and $R_{N}R_{N}^{\mathrm{T}}=I_{N}-r_{N}r_{N}^{\mathrm{T}}$.

2 Problem formulation

Considering a collection of multi-agent systems as follows:

[TABLE]

where $x_{i}\in\mathbb{R}^{n_{i}}$, $u_{i}\in\mathbb{R}$, and $y_{i}\in\mathbb{R}$ are its state, input, and output variables of agent $i$. The signal $\omega_{i}\in\mathbb{R}^{q_{i}}$ is an external disturbance of agent $i$ modeled by

[TABLE]

Associated with these agents, we endow agent $i$ with a local cost function $f_{i}\colon\mathbb{R}\to\mathbb{R}$. For the whole multi-agent system, we assume its prescribed steady-state is determined as the optimal solution of the following constrained optimization problem:

[TABLE]

where $y\triangleq\mbox{col}(y_{1},\,\dots,\,y_{N})$, the function $f_{i}(\cdot)$ and constant $d_{i}$ are private to agent $i$ and can not be shared with other agents.

This optimization problem is often called resource allocation and many practical applications can be formulated as the above, e.g. economic dispatch in power systems [19], flow control in networks [21]. Coupled with the physical agents, we aim to design proper controllers such that the outputs of these agents asymptotically solve the optimization problem (3). In other words, the controllers should regulate the agents’ outputs such that the equality constraint is satisfied and optimal performance is achieved, both in an asymptotic manner.

Without loss of generality, we assume the triple $(C_{i},\,A_{i},\,B_{i})$ is minimal and the pair $(E_{i},\,S_{i})$ observable. If not, we can always define a new minimal triple $(C_{i},\,A_{i},\,B_{i})$ and observable pair $(E_{i},\,S_{i})$ by removing the undesired modes of systems (1) and (2), which will in no way affect the designed goal[22].

Due to the privacy of local cost function $f_{i}(\cdot)$ and resource data $d_{i}$, each agent can only have a part of the global optimization problem and no agent can gather enough information to determine the optimal solution by itself. Hence, the steady-state can not be obtained off-line without on-line cooperation and information sharing among these agents.

We use an undirected graph $\mathcal{G}=(\mathcal{N},\mathcal{E},\mathcal{A})$ to describe the information sharing topology with the node set $\mathcal{N}=\{1,{\dots},N\}$ and the edge set $\mathcal{E}$ [23]. An edge $(i,\,j)\in\mathcal{E}$ between nodes $i$ and $j$ means that agent $i$ and agent $j$ can share information with each other. The weighted adjacency matrix $\mathcal{A}=[a_{ij}]\in\mathbb{R}^{N\times N}$ is defined by $a_{ii}=0$ and $a_{ij}=a_{ji}\geq 0$ ($a_{ij}>0$ if and only if there is an edge between node $i$ and node $j$). The Laplacian $L=[l_{ij}]\in\mathbb{R}^{N\times N}$ of graph $\mathcal{G}$ is defined as $l_{ii}=\sum_{j\neq i}a_{ij}$ and $l_{ij}=-a_{ij}(j\neq i)$, which is thus symmetric.

Regarding multi-agent system (1), external disturbance (2) and an information sharing graph $\mathcal{G}$, the distributed optimal steady-state regulation problem is to find distributed controllers for each agent by using its local data and exchanged information with its neighbors, such that its output $y_{i}(t)$ converges to a steady-state $y_{i}^{*}$ as $t$ goes to infinity for $i=1,\,\dots,\,N$, where $\mbox{col}(y_{1}^{*},\,\dots,\,y_{N}^{*})$ is an optimal solution of problem (3).

Remark 1**.**

This problem can be taken as a distributed resource allocation problem for heterogeneous high-order agents subject to external disturbances, and thus includes the formulation for single integrators as a special case [19, 20]. Particularly, when $f_{i}(y_{i})=\frac{1}{2}y_{i}^{2}$ and $d_{i}=y_{i}(0)$, the optimal solution of (3) is $\frac{\sum_{i=1}^{N}y_{i}(0)}{N}{\bf 1}_{N}$. Thus, this formulation provides another way to achieve an average consensus [24] for general liner agents subject to external disturbances.

To solve the formulated problem, we need several technical assumptions, which are very standard in coordination of multi-agent systems [20, 19, 3, 25].

Assumption 1**.**

The graph $\mathcal{G}$ is connected.

Assumption 2**.**

[TABLE]

It is well-known that under Assumption 1, the associated Laplacian $L$ of this graph is positive semidefinite with rank $N-1$, and its null space is spanned by $\bm{1}_{N}$. Assumption 2 is made to guarantee the wellposedness of optimization problem (3). In fact, it implies strong smoothness and strong convexity of $f_{i}(\cdot)$. Then, the problem (3) is solvable and has a unique solution $y^{*}=\mbox{col}(y_{1}^{*},\,\dots,\,y_{N}^{*})$ by the refined Slater’s condition [26]. As usual, we assume $y^{*}$ is finite [19, 25].

Assumption 3**.**

For each $i=1,\,\dots,\,N$, there exist constant matrices $X_{i1}$, $X_{i2}$, $U_{i1}$ and $U_{i2}$ satisfying

[TABLE]

and

[TABLE]

Remark 2**.**

This assumption is known as the solvability of regulator equations and plays a key role in achieving the asymptotic steady-state regulation goal [27, 28]. A sufficient condition to guarantee this assumption is that

[TABLE]

When agent $i$ satisfies the matching condition, i.e., $B_{i}M_{i}=E_{i}$ for some matrix $M_{i}$, the first equation naturally holds by letting $X_{i1}=0$ and $U_{i1}=-M_{i}$. Moreover, for minimum-phase agents, the second equation is also fulfilled, including integrators as special cases.

3 Embedded Control and Problem Conversion

To avoid the difficulties brought by the high-order structure of agents, we employ the embedded scheme proposed in [16] to solve the formulated distributed optimal steady-state regulation problem.

In this embedded control scheme, we first introduce an optimal signal generator by considering the same optimization problem for “virtual” single integrators, in order to asymptotically reproduce the optimal solution $y^{*}$ by a vector $z\triangleq\mbox{col}(z_{1},\,\dots,\,z_{N})$. Then, by taking $z_{i}$ as an output reference signal for agent $i$, we embed this generator in the feedback loop via a tracking controller for system (1). In this way, we can divide the distributed optimal steady-state regulation problem into two simpler subproblems, i.e., optimal signal generator construction and asymptotic tracker design. The former is to solve the distributed resource allocation problem for single integrators and has been studied by many authors [20, 19], while the latter is an asymptotic tracking problem for general linear agents of the form (1) subject to external disturbance generated by (2).

To give a self-contained design, we consider the following algorithm proposed in [29]:

[TABLE]

where $\lambda^{0}_{i}\triangleq\sum\nolimits_{j=1}^{N}a_{ij}(\lambda_{i}-\lambda_{j})$, $v^{0}_{i}\triangleq\sum\nolimits_{j=1}^{N}a_{ij}(v_{i}-v_{j})$.

This algorithm is inherently a primal-dual dynamics to solve the problem (3) and has also been partially investigated in [19]. The following lemma shows the feasibility of this dynamics as a distributed optimal signal generator for our problem.

Lemma 1**.**

Suppose Assumptions 1–2 hold. Then, the algorithm (4) will exponentially generate the optimal solution of the distributed optimization problem (3), i.e., $z_{i}$ converges to $y^{*}_{i}$ exponentially fast as $t\to+\infty$ for $i=1,\,\ldots,\,N$.

**Proof. ** First, we show that at the equilibrium point $(z_{i}^{*},\,\lambda_{i}^{*},\,v_{i}^{*})$ of system (4), it solves the optimization problem (3). For this purpose, we let the righthand side of (4) be zero and have $-\nabla f_{i}(z_{i})+\lambda_{i}=0$,  $-\lambda^{0}_{i}-v^{0}_{i}+d_{i}-z_{i}=0$,  $\lambda^{0}_{i}=0$. By summing the second equation up from $1$ to $N$, it follows $-\sum\nolimits_{j=1}^{N}v^{0}_{i}+\sum\nolimits_{j=1}^{N}d_{i}-\sum\nolimits_{j=1}^{N}z_{i}=0$. Under Assumption 1, we have ${\bm{1}}_{N}^{\mathrm{T}}L=0$, which implies that $\sum\nolimits_{j=1}^{N}v^{0}_{i}={\bm{1}}_{N}^{\mathrm{T}}Lv=0$ and thus $\sum\nolimits_{j=1}^{N}d_{i}-\sum\nolimits_{j=1}^{N}z_{i}=0$. Moreover, from $\lambda^{0}_{i}=0$ ($i=1,\,\dots,\,N$), we have $L\lambda=\mbox{col}(\lambda^{0}_{1},\,\dots,\,\lambda^{0}_{N})={\bm{0}}_{N}$. This implies that there exists a constant $\theta^{*}$ satisfying $\lambda_{1}=\dots=\lambda_{N}=\theta^{*}$ since the kernel of graph Laplacian $L$ is spanned by ${\bm{1}}_{N}$. The above arguments imply that at the equilibrium point $(z_{i}^{*},\,\lambda_{i}^{*},\,v_{i}^{*})$, it must hold that $\sum\nolimits_{j=1}^{N}d_{i}-\sum\nolimits_{j=1}^{N}z^{*}_{i}=0$ and $\nabla f_{1}(z_{1}^{*})=\dots=\nabla f_{N}(z_{N}^{*})$. Then, we recall the refined Slater’s condition [26] and conclude that $\mbox{col}(z_{1}^{*},\,\dots,\,z_{N}^{*})$ indeed solves the convex optimization problem (3). From the uniqueness of this optimal solution to (3), one can further have $y_{i}^{*}=z_{i}^{*}$ for $i=1,\,\dots,\,N$.

Next, we consider the exponential stability of (4) at the equilibrium $(z_{i}^{*},\,\lambda_{i}^{*},\,v_{i}^{*})$. Performing a coordinated transformation $\bar{z}_{i}=z_{i}-z_{i}^{*}$, $\bar{\lambda}_{i}=\lambda_{i}-\lambda_{i}^{*}$, $\bar{v}_{i}=v_{i}-v_{i}^{*}$ gives that

[TABLE]

where ${\bm{h}}_{i}(\bar{z}_{i},\,z_{i}^{*})\triangleq\nabla f_{i}(z_{i})-\nabla f_{i}(z_{i}^{*})$, $\bar{\lambda}^{0}_{i}\triangleq\sum\nolimits_{j=1}^{N}a_{ij}(\bar{\lambda}_{i}-\bar{\lambda}_{j})$, $\bar{v}^{0}_{i}\triangleq\sum\nolimits_{j=1}^{N}a_{ij}(\bar{v}_{i}-\bar{v}_{j})$. Denote $\bar{z}=\mbox{col}(\bar{z}_{1},\,\dots,\,\bar{z}_{N})$, $\bar{\lambda}=\mbox{col}(\bar{\lambda}_{1},\,\dots,\,\bar{\lambda}_{N})$ and $\bar{v}=\mbox{col}(\bar{v}_{1},\,\dots,\,\bar{v}_{N})$. It can be written into a compact form:

[TABLE]

where the vector-valued function ${\bm{h}}(\bar{z},\,z^{*})$ is determined by ${\bm{h}}_{i}(\bar{z}_{i},\,z_{i}^{*})$. By Assumption 2, ${\bm{h}}(\bar{z},\,z^{*})$ is $\bar{h}$-Lipschitz in $\bar{z}$ and satisfies $\bar{z}^{\mathrm{T}}{\bm{h}}(\bar{z},\,z^{*})\geq\underline{h}\bar{z}^{\mathrm{T}}\bar{z}$ for $\overline{h}=\max_{i}\{\overline{h}_{i}\}$ and $\underline{h}=\min_{i}\{\underline{h}_{i}\}$.

At present, we only have to investigate the stability of the above system at the origin. For this purpose, let $\hat{v}_{1}=r^{\mathrm{T}}\bar{v}$, $\hat{v}_{2}=R^{\mathrm{T}}\bar{v}$. Since $r^{\mathrm{T}}\dot{v}=0$, it follows that $\dot{\hat{v}}_{1}=r^{\mathrm{T}}\dot{\bar{v}}=0$. This means $r^{\mathrm{T}}v(t)$ is an invariant quantity and thus has no effect on the stability of the above system. Furthermore, the rest states of this system can be put into the following form:

[TABLE]

where we use $Lr=0$ and $L=L^{\mathrm{T}}$ by Assumption 1.

Let $T\triangleq\begin{bmatrix}-L&-LR\\ R^{\mathrm{T}}L^{\mathrm{T}}&{\bm{0}}\end{bmatrix}$ and $S\triangleq\begin{bmatrix}-I_{N}\\ {\bm{0}}\end{bmatrix}^{\mathrm{T}}$. It can be easily verified that $T+T^{\mathrm{T}}\leq{\bm{0}}$ and the pair $(S,\,T)$ is observable. Then, we apply Lemma 2 in [16] to the above system and conclude its exponential stability at the origin, which implies the conclusions. The proof is thus complete.

With the designed optimal signal generator, our formulated distributed optimal steady-state regulation problem is converted to an asymptotic tracking and disturbance rejection problem for linear multi-agent systems of the form (1) subject to external disturbances generated by (2).

4 Solvability of Optimal Steady-State Regulation

In this section, we seek proper tracking controllers for agent (1) with output reference $z_{i}$, and finally solve the associated optimal steady-state regulation problem.

Note that $(A_{i},\,B_{i})$ is controllable, there exists $K_{i1}$ such that the matrix $A_{i}+B_{i}K_{i1}$ is Hurwitz. Denote $K_{i2}\triangleq U_{i1}-K_{i1}X_{i1}$, $K_{i3}\triangleq U_{i2}-K_{i1}X_{i2}$. From the conventional output regulation theory (e.g., Theorem 1.7 in [28]), a full-information regulator $u^{o}_{i}=K_{i1}x_{i}+K_{i2}\omega_{i}+K_{i3}y_{i}^{*}$ can steer agent $i$ to its associated steady-state and also achieve disturbance rejection. As the optimal signal generator (4) can exponentially generate $y_{i}^{*}$, we only have to consider the disturbance rejection part.

Inspired by the techniques used in [28, 30], we construct a reduced-order observer to estimate the disturbance $\omega_{i}$.

[TABLE]

where $\bar{L}_{i}$ is chosen such that the matrix $S_{i}-\bar{L}_{i}E_{i}$ is Hurwitz. Its existence is guaranteed by the observability of $(E_{i},\,S_{i})$.

Here is a lemma to show the effectiveness of this estimator.

Lemma 2**.**

Consider the dynamic system composed of (1) and (5). Then, $\eta_{i}(t)$ exponentially converges to $\omega_{i}$, i.e., there exist two positive constants $\mu_{1},\,\mu_{2}$ satisfying $||\eta_{i}(t)-\omega_{i}(t)||\leq\mu_{1}e^{-\mu_{2}t}$ for all $t>0$.

**Proof. ** To prove this lemma, we let $\hat{\eta}_{i}\triangleq\eta_{i}-\omega_{i}$. Then, along the trajectory of systems (2) and (5), we have

[TABLE]

By the selection of $\bar{L}_{i}$, the matrix $(S_{i}-\bar{L}_{i}E_{i})$ is Hurwitz. In other words, this error system is exponentially stable. This implies the existence of $\mu_{1},\,\mu_{2}>0$ satisfying this lemma.

With this estimator, we substitute it into the full-information regulator and propose a state-feedback controller for agent (1) to solve our formulated problem as follows.

Theorem 1**.**

Suppose Assumptions 1–3 hold. Then, the distributed optimal steady-state regulation problem for multi-agent system (1) with optimization problem (3) and disturbance (2) is exponentially solved by the following control:

[TABLE]

where $\eta_{i}=\bar{\eta}_{i}+\bar{L}_{i}x_{i}$, $K_{i1},\,K_{i2},\,K_{i3}$ and $\bar{L}_{i}$ are chosen gain matrices as above.

**Proof. ** To prove this theorem, we first perform a coordination transformation $\bar{x}_{i}=x_{i}-X_{i1}\omega_{i}-X_{i2}z_{i}$ and $\hat{\eta}_{i}=\eta_{i}-\omega_{i}$. By Assumption 3 and some mathematical manipulations, the whole composite system can be put into the following form:

[TABLE]

where $\bar{y}_{i}\triangleq y_{i}-z_{i}$.

By the selection of $K_{i1}$, the matrix $\bar{A}_{i}\triangleq A_{i}+B_{i}K_{i1}$ is Hurwitz. Thus, the Lyapunov equation $\bar{A}_{i}^{\mathrm{T}}P_{i}+P_{i}\bar{A}_{i}=-2I_{n_{i}}$ has a unique positive definite solution $P_{i}$. Similarly, we can determine a unique positive definite matrix $Q_{i}$ satisfying $(S_{i}-\bar{L}_{i}E_{i})^{\mathrm{T}}Q_{i}+Q_{i}(S_{i}-\bar{L}_{i}E_{i})=-I_{q_{i}}$.

Considering the first two subsystems, we let $V^{0}_{i}(\bar{x}_{i},\,\hat{\eta}_{i})=\bar{x}_{i}^{\mathrm{T}}P_{i}\bar{x}_{i}+c_{i}\hat{\eta}_{i}^{\mathrm{T}}Q_{i}\hat{\eta}_{i}$ with a constant $c_{i}>0$ to be specified later. Its time derivative along the trajectory of (7) is:

[TABLE]

By Young’s inequality, it follows that

[TABLE]

Letting $c_{i}>2||P_{i}B_{i}K_{i2}||^{2}+1$ implies

[TABLE]

We let $V^{0}=\sum\nolimits_{i=1}^{N}V_{i}^{0}$ and further have

[TABLE]

Choosing $c=1+2\max\limits_{i}\{||P_{i}X_{i2}||^{2}\}$ gives

[TABLE]

Next, we consider the optimal signal generator part. From the proof of Lemma 1, the following system is exponentially stable at the origin.

[TABLE]

Denote $\hat{z}\triangleq\mbox{col}(\bar{z},\,\bar{\lambda},\,{\hat{v}}_{2})$ for short. Since ${\bm{h}}(\bar{z},\,z^{*})$ is global Lipschitz for some $\hat{l}>1$ in $\bar{z}$ by Assumption 2, invoking the converse Lyapunov theorem (Theorem 4.15 in [27]) provides us a Lyapunov function $W(\hat{z})$ satisfying the following conditions for some positive constants $\hat{c}_{1}$, $\hat{c}_{2}$, $\hat{c}_{3}$, $\hat{c}_{4}$:

[TABLE]

Since $\dot{z}=\dot{\bar{z}}$, we choose a Lyapunov function for the composite system as $V(\bar{x},\,\hat{\eta},\,\hat{z})=V^{0}(\bar{x},\,\hat{\eta})+\hat{c}W(\hat{z})$. Its time derivative along the trajectory of the closed-loop system satisfies

[TABLE]

where we use the $\hat{l}$-Lipschitzness of $\dot{z}$.

Letting $\hat{c}\geq\frac{\hat{c}_{2}(\hat{c}_{1}+c\hat{l}^{2})}{\hat{c}_{1}\hat{c}_{3}}$ gives

[TABLE]

Recalling the quadratic form of $V^{0}$ and the properties of $W$, we invoke Theorem 4.10 in [27] and conclude the global exponential convergence of the whole composite system with respect to its equilibrium point.

From the above arguments, we have that a) $z_{i}$ exponentially converges to $y_{i}^{*}$ as $t\to\infty$ and b) $y_{i}$ exponentially converges to $z_{i}$ as $t\to\infty$. It follows by combing these two facts that

[TABLE]

Moreover, the convergence is also exponentially fast. The proof is thus complete.

Remark 3**.**

Compared with existing steady-state regulation problem [17, 18], we consider its distributed extension for a multi-agent system where the steady-state can only be determined and reached in a distributed way, which is of course more challenging. When agents are all single integrators, the above results are consistent with existing distributed resource allocation conclusions [20, 19]. Even in this case, we consider unknown (unbounded) disturbances, while most of relevant publications are disturbance-free.

In some cases, only the output variables of each agent can be obtained because it may be difficult to get or measure all the state variables in some situations. Since the optimal signal generator is independently implemented, we only have to focus on the tracking part to solve the optimal regulation problem with output feedbacks.

By attaching an observer to the above embedded controller (6), the following theorem is readily obtained.

Theorem 2**.**

Suppose Assumptions 1–3 hold. Then, the distributed optimal steady-state regulation problem for multi-agent system (1) with optimization problem (3) and disturbance (2) is exponentially solved by the following control:

[TABLE]

where the gain matrices $K_{i1}$, $K_{i2}$, $K_{i3}$, $\bar{L}_{i}$ are defined as in controller (6), $\hat{L}_{i}$ is chosen such that $A_{i}+\hat{L}_{i}C_{i}$ is Hurwitz.

The proof is similar to that of Theorem 1 and thus omitted.

5 Extension with Real-time Gradients

In many practical cases, we can not have the cost function $f_{i}(\cdot)$ itself and only its real-time gradient $\nabla f_{i}(y_{i})$ is available. Then, the above control laws will not be implementable.

Note that there will always be some error between $\nabla f_{i}(y_{i})$ and $\nabla f_{i}(z_{i})$ when $z_{i}\neq y_{i}$ by Assumption 2, and this error will be smaller if the tracking subsystem evolves in a faster time scale. For this purpose, we focus on minimum-phase agents of the form (1) with relative degree $r_{i}>0$, and use a high-gain rule to choose the gain matrices. We only consider the state feedback case here, while its output feedback extension can be derived without difficulties and thus omitted.

Let $\bar{X}_{i1,1}=0$, $\bar{X}_{i1,k+1}=\bar{X}_{i1,k}S_{i}-C_{i}A_{i}^{k-1}E_{i}$ for $k=1,\,\dots,\,r_{i}-1$. Denote $\bar{X}_{i1}=\mbox{col}(\bar{X}_{i1,1},\,\dots,\,\bar{X}_{i1,r_{i}})$, $\bar{U}_{i1}\triangleq\frac{\bar{X}_{i1,r_{i}}S_{i}-C_{i}A_{i}^{r_{i}-1}E_{i}}{C_{i}A_{i}^{r_{i}-1}B_{i}}$, $\bar{X}_{i2}\triangleq\mbox{col}(1,\,0,\,\cdots,\,0)$, $U_{i2}\triangleq 0$, $\bar{K}_{i1}\triangleq-\frac{1}{C_{i}A_{i}^{r_{i}-1}B_{i}\varepsilon^{r_{i}}}[c_{i0}~{}\varepsilon c_{i1}~{}\cdots~{}\varepsilon^{r_{i}-1}c_{ir_{i}}]$, $\bar{K}_{i2}\triangleq\bar{U}_{i1}-\bar{K}_{i1}\bar{X}_{i1}$, $K_{i3}\triangleq\bar{U}_{i2}-\bar{K}_{i1}\bar{X}_{i2}$, and $\hat{X}_{i}\triangleq\mbox{col}(C_{i},\,C_{i}A_{i},\,\dots,\,C_{i}A_{i}^{r_{i}-1})$ with constants $\varepsilon,\,c_{i0},\,\dots,\,c_{ir_{i}-1}$ to be specified later.

The main result with real-time gradients is as follows.

Theorem 3**.**

Suppose Assumptions 1–2 hold and agent (1) is minimum-phase. Then, there exists a constant $\varepsilon^{*}>0$ such that the distributed optimal steady-state regulation problem for multi-agent system (1) with optimization problem (3) and disturbance (2) is exponentially solved by the following control with any $\varepsilon\in(0,\,\varepsilon^{*})$.

[TABLE]

where the matrix $\bar{L}_{i}$ is defined as above and the constants $c_{i0},\dots,\,c_{ir_{i}-1}$ are chosen such that the polynomial $\sum_{k=0}^{r_{i}-1}c_{ik}s^{k}+s^{r_{i}}$ is Hurwitz, $i=1,\,\dots,\,N$.

**Proof. ** First, we put the original dynamics into a normal form. Note that the plant (1) has a relative degree $r_{i}$. When $r_{i}\geq 2$, we can always define a coordinate transformation and put system (1) into the following form:

[TABLE]

where $\chi_{ik}=C_{i}A_{i}^{k-1}x_{i}$ and $\chi_{i}^{z}$ is the rest state variable. When $r_{i}=1$, the transformed system is described by a similar form:

[TABLE]

To simplify the following proof, we only consider the case when $r_{i}\geq 2$ without loss of generality. The arguments for $r_{i}=1$ is almost the same and thus omitted.

From the minimum-phase assumption, matrix $A_{i}^{0}$ is Hurwitz, which implies that the trajectory of $\chi^{z}_{i}$ is always well-defined. By letting $u_{i}=\bar{u}_{i}-\frac{C_{i}A_{i}^{r_{i}}}{C_{i}A_{i}^{r_{i}-1}B_{i}}x_{i}$, the $\chi^{z}_{i}$-subsystem will not effect the output of this plant. Thus, we only have to consider the $\chi_{i}$-subsystem for agent $i$ with this new input as follows:

[TABLE]

Next, we solve the distributed optimal steady-state regulation problem for the above agents with disturbance (2) and optimization problem (3).

Considering the closed-loop system, we substitute the control (3) into each agent and define a coordinate transformation $\bar{\chi}_{i}=\chi_{i}-\bar{X}_{i1}\omega_{i}-\bar{X}_{i2}z_{i}$. It follows that

[TABLE]

Letting $\hat{\chi}_{i1}=\bar{\chi}_{i1}$ and $\hat{\chi}_{ik}=\varepsilon^{k-1}\bar{\chi}_{ik}$ ($k=2,\,\dots,\,r_{i}$) gives

[TABLE]

where A_{ic}={\left[\begin{array}[]{c|c}0&\mathrm{I}_{r_{i}-1}\\ \hline\cr c_{i0}&c_{i1}\,\dots\,c_{ir_{i}-1}\end{array}\right]} and $B_{ic}=\mbox{col}(1,\,0,\,\dots,\,0)$. Putting all agents in a compact form gives

[TABLE]

where $\hat{\chi}\triangleq\mbox{col}(\hat{\chi}_{1},\,\dots,\,\hat{\chi}_{N})$,  $z\triangleq\mbox{col}(z_{1},\,\dots,\,z_{N})$,  $A_{c}\triangleq\mbox{block}\,diag\{A_{1c},\,\dots,\,A_{Nc}\}$ and $B_{c}\triangleq\mbox{block}\,diag\{B_{1c},\,\dots,\,B_{Nc}\}$. By constructions, $A_{ic}$ and $A_{c}$ are Hurwitz.

By similar arguments in the proof of Lemma 1, the optimal signal generator part can be rewritten as follows:

[TABLE]

where $\bar{z}=z-y^{*}$ and $\Delta(z,y)$ is a vector-value function defined as follows:

[TABLE]

By Assumption 2, $\Delta(z,y)$ is global Lipschitz in $\bar{y}$ and then $\hat{\chi}$.

Note that when $y=z$, we have ${\bm{h}}(y,\,z)=0$ and the above system is exponentially stable at the origin by Lemma 1. Since $\dot{z}=\dot{\bar{z}}$, it implies that $\dot{z}$ is linearly upper bounded by ${\mathrm{c}ol}(\bar{z},\,\bar{\lambda},\,\hat{\chi})$. Then, the whole system composed of (10) and (11) is in a singularly perturbed form. By Corollary 2.3 in Chapter 7 of [31], we can obtain the conclusions.

Remark 4**.**

The embedded control approach used here was first proposed in [16] to solve an optimal output consensus problem of linear multi-agent systems. Here, we extend it to tackle a much different problem, where the global optimization problem is constrained and the agents are subject to external disturbances. With this approach, the design complexities brought by high-order dynamics and disturbance direction are decoupled from the optimization task. Then, the optimal steady-state regulation problem is solved in a constructive way by solving two simpler subproblems, which verifies the flexibility and effectiveness of this embedded design.

6 Simulations

In this section, we provide some numerical examples to illustrate the effectiveness of previous designs.

Example 1. Consider an optimal rendezvous problem of wheeled robots with the following dynamics.

[TABLE]

where $(r_{i}^{x},\,r_{i}^{y},\,\theta_{i})$ are the inertia center’s position and orientation of the $i$th robot, $(v_{i},\,\omega_{i})$ the linear and angular speed, $(F_{i},\,\tau_{i})$ the applied force and torque, $(m_{i},\,J_{i})$ the mass and moment of inertia for $i=1,\,\dots,\,4$. Let $c_{i}$ represent the distance between the hand position and inertia center of the $i$th robot. Following the arguments in [1], we employ an output feedback linearization technique about the hand position $\tilde{x}_{i}\triangleq(r_{i}^{x}+c_{i}\cos(\theta_{i}),\,r_{i}^{y}+c_{i}\sin(\theta_{i}))$ and obtain a simple linear dynamics as follows:

[TABLE]

To drive all hands of robots to rendezvous at a common point that minimizes the aggregate distance from their starting points to this final location, we formulate this problem as an optimal steady-state regulation problem by taking the cost functions as $f_{i}(y_{i})=\frac{1}{2}||y_{i}||^{2}$ and $d_{i}=y_{i}(0)$ ($i=1,\,\dots,\,4$). We can easily check that the optimal solution of the global cost function is $y^{*}=\frac{\sum_{i=1}^{4}y_{i}(0)}{4}{\bm{1}}_{4}$.

The information sharing graph satisfying Assumption 1 is chosen as Fig. 1 with unity edge weights. Choose constants $c_{i0}$ and $c_{i1}$ such that the polynomial $s^{2}+c_{i1}s+c_{i0}$ is Hurwitz. Then, the optimal rendezvous problem is solved according to Theorem 1 by a control as follows:

[TABLE]

Take $c_{i0}=4,\,c_{i1}=8,\,\varepsilon=1$ with initials (randomly) chosen in $[-10,\,10]$. The simulation result is given in Fig. 2 and all robots achieve the optimal rendezvous at $\frac{\sum_{i=1}^{4}y_{i}(0)}{4}$.

Example 2. Another example is the distributed inventory control problem with unknown demand rates. Consider a network of inventories which produce only one commodity [32]. The inventory system at node $i$ can be modeled as

[TABLE]

where $I_{i}$, $P_{i}$ and $D_{i}$ are the inventory level, production rate and demand rate at node $i$. The storage cost at each warehouse is given as $f_{i}(I_{i})=\alpha_{i}I_{i}^{2}+\beta_{i}I_{i}+\gamma_{i}$, where $\alpha_{i}>0$. In distributed inventory control, we aim to maintain the total inventory at certain level $I^{r}$ to satisfy the customer’s demands and some safety goals. To make it more interesting, we consider the case when the demand rate $D_{i}$ is constant but unknown.

This problem can be formulated as a distributed optimal steady-state regulation problem for inventory systems with external disturbances, while the optimal steady-state is determined by the following problem.

[TABLE]

Clearly, with a preallocation of inventory level $I^{r}=\sum\nolimits_{i=1}^{N}I_{i}^{r}$, all assumptions are fulfilled to apply the control (6).

For simulations, we take $N=4,\,\alpha_{i}=0.1i,\,\beta_{i}=-0.05i,\,\gamma_{i}=D_{i}=I_{i}^{r}=i$, $i=1,\,\dots,\,4$. The communication graph is still taken as Fig. 1 with initials picked in $[0,\,6]$. By choosing proper gain matrices, we solve this problem and drive the inventory levels of all agents to the optimal solution $I^{*}=\mbox{col}(4.57,\,2.41,\,1.69,\,1.33)$ as depicted in Fig. 3.

Example 3. Consider a distributed coordination problem for a multi-agent system consisting of four mass-damper-spring systems with unit mass described by:

[TABLE]

where $\omega_{i}$ is the local disturbance. These disturbances are modeled by $S_{1}=[0~{}1;\,0~{}0]$, $F_{1}=[1~{}0]$, $S_{2}=0$, $F_{2}=1$, $S_{3}=[0~{}1;\,-1~{}0]$, $F_{3}=[1~{}0]$, $S_{4}=[0~{}2;\,-2~{}0]$, $F_{4}=[1~{}0]$. The local cost functions are chosen as $f_{1}(y_{1})=\frac{1}{2}(y_{1}+2)^{2}$, $f_{2}(y_{2})=y_{2}^{2}\ln(1+y_{2}^{2})+(y_{2}+1)^{2}$, $f_{3}(y_{3})=\ln(e^{-0.1y_{3}}+e^{0.3y_{3}})+y_{3}^{2}$, $f_{4}(y_{4})=\frac{y_{4}^{2}}{25\sqrt{y_{4}^{2}+1}}+(y_{4}-3)^{2}.$ Set $d_{i}=i$ and graph $\mathcal{G}$ as in Fig. 1. The optimal solution is $y^{*}\approx\mbox{col}(2.2,\,0.7,\,2.0,\,5.1)$.

Letting $x_{i1}=y_{i},\,x_{i2}=\dot{y}_{i}$ gives

[TABLE]

It can be verified that Assumptions 1–3 are all satisfied. Thus, the associated distributed optimal steady-state regulation problem can be solved by the controls given in last sections.

For simulations, we assume only real-time gradients are available. The system parameters are taken as $f_{1}=1,\,g_{1}=1,\,f_{2}=0,\,g_{2}=1$,  $f_{3}=1,\,g_{3}=0$,  $f_{4}=0,\,g_{4}=0$. Then, we can solve the regulator equations in Assumption 3 and choose proper gain matrices for the state feedback control (3). To verify disturbance rejection performance, we first simulate the disturbance-free dynamics during $t=0\sim 40\,{\mathrm{s}}$, and then the disturbed case after $t=40\,{\mathrm{s}}$. After the disturbance rejection part works at $t=60\,{\mathrm{s}}$, the optimal steady-state is quickly recovered. The simulation result is showed in Fig. 4. Satisfactory regulating performance is observed.

7 Conclusions

A distributed optimal steady-state regulation problem was formulated for general linear agents as a combination of asymptotic steady-state regulation and distributed resource allocation. By an embedded control scheme, we introduced an optimal signal generator and divided this problem into two simpler subproblems. Both state and output feedback cases were investigated and solved with disturbance rejection. A nontrivial extension using only real-time gradient information was also presented by high-gain control techniques. Future works will include nonlinear agents with nonsmooth cost functions.