Explicit Agent-Level Optimal Cooperative Controllers for Dynamically Decoupled Systems with Output Feedback

TL;DR

This paper develops explicit, optimal output-feedback controllers for a network of agents with decoupled dynamics, leveraging the graph structure to enable cooperative optimization of a global objective.

Contribution

It introduces explicit state-space formulations for agent-level controllers in a networked setting with output feedback, emphasizing the role of the directed acyclic graph structure.

Findings

Provides minimal agent-level controller implementations

Highlights the importance of graph structure in controller design

Offers intuitive interpretations of controller states and information flow

Abstract

We consider a dynamically decoupled network of agents each with a local output-feedback controller. We assume each agent is a node in a directed acyclic graph and the controllers share information along the edges in order to cooperatively optimize a global objective. We develop explicit state-space formulations for the jointly optimal networked controllers that highlight the role of the graph structure. Specifically, we provide generically minimal agent-level implementations of the local controllers along with intuitive interpretations of their states and the information that should be transmitted between controllers.

Equations

$${\dot{x}}_i$$

$$y_i$$

$$\mathcal{J}=\lim_{T\to\infty}\frac{1}{T}\operatorname{\mathbf{E}}\int_{0}^{T}\bigl{\lVert}{C_{1}x(t)+D_{12}u(t)}\bigr{\rVert}^{2}\,\mathrm{d}t,$$

$$\displaystyle\mathcal{G}(s)=\left[\begin{array}[]{c|c}A&B\\ \hline\cr\rule{0.0pt}{11.19443pt}C&D\end{array}\right]=D+C(sI-A)^{-1}B.$$

$$\displaystyle\left[\begin{array}[]{c c}\mathcal{U}&\mathcal{V}\\ \mathcal{W}&\mathcal{G}\end{array}\right]\mathrel{\mathchoice{\vbox{\hbox{$\displaystyle:$}}}{\vbox{\hbox{$\textstyle:$}}}{\vbox{\hbox{$\scriptstyle:$}}}{\vbox{\hbox{$\scriptscriptstyle:$}}}{=}}\left[\begin{array}[]{c|cc}A&B_{1}&B_{2}\\ \hline\cr\rule{0.0pt}{11.19443pt}C_{1}&0&D_{12}\\ C_{2}&D_{21}&0\end{array}\right].$$

$$\underset{\mathcal{K}\text{stabilizes}\mathcal{G}}{\mathrm{minimize}}$$

$$\mathcal{K}\in {\mathbb{S}}_{{\mathcal{R}}_{\text{p}}}.$$

$$\displaystyle\mathcal{P}_{\underline{i}i}=\left[\begin{array}[]{c|c}A_{\underline{ii}}+B_{2_{\underline{ii}}}F^{i}+\begin{bmatrix}L^{i}\\ 0\end{bmatrix}C_{2_{i\underline{i}}}&\begin{bmatrix}-L^{i}\\ 0\end{bmatrix}\\ \hline\cr\rule{0.0pt}{11.19443pt}F^{i}&0\end{array}\right]$$

$$(X^i,F^i)$$

$$(Y^i,{L^i}^{\mathsf{T}})$$

$$\displaystyle\mathcal{K}=\left[\begin{array}[]{c|c}\begin{matrix}\bar{A}+\bar{L}\bar{C}+\bar{B}\bar{S}_{m}\bar{F}\bar{S}_{n}^{-1}\end{matrix}&\begin{matrix}{-\bar{L}\bar{\mathbf{1}}_{p}}\end{matrix}\\ \hline\cr\rule{0.0pt}{11.19443pt}\begin{matrix}{\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F}\bar{S}_{n}^{-1}}\end{matrix}&0\end{array}\right],$$

$$\bar{A}\text{vbox}:=I_N\otimes A,\bar{B}\text{vbox}:=I_N\otimes B_2,\bar{C}\text{vbox}:=I_N\otimes C_2,$$

$${\bar{S}}_m\text{vbox}:=S\otimes I_m,{\bar{S}}_n\text{vbox}:=S\otimes I_n,{\bar{S}}_p\text{vbox}:=S\otimes I_p,$$

$${\bar{1}}_m\text{vbox}:=1_N\otimes I_m,{\bar{1}}_p\text{vbox}:=1_N\otimes I_p.$$

$$L^{\bar{i}}\text{vbox}:=\mathrm{blkdiag}(\{L^j{\}}_{j\in \bar{i}})$$

$$\bar{L}\text{vbox}:=\mathrm{blkdiag}(\{L_i\})\text{where: }{L}_i\text{vbox}:=E_{n_{\bar{i}}}{L}^{\bar{i}}{E}_{p_{\bar{i}}}^{\mathsf{T}}$$

$$\bar{F}\text{vbox}:=\mathrm{blkdiag}(\{F_i\})\text{where: }{F}_i\text{vbox}:=E_{m_{\underline{i}}}{F}^i{E}_{n_{\underline{i}}}^{\mathsf{T}}.$$

$$\displaystyle\mathcal{K}={\left[\begin{array}[]{c|c}\begin{matrix}\bar{A}+\bar{S}_{n}^{-1}\bar{L}\bar{S}_{p}\bar{C}+\bar{B}\bar{F}\end{matrix}&\begin{matrix}-\bar{S}_{n}^{-1}\bar{L}\bar{\mathbf{1}}_{p}\end{matrix}\\ \hline\cr\rule{0.0pt}{11.19443pt}{\begin{matrix}\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F}\end{matrix}}&0\end{array}\right]},$$

$$\dot{\xi }$$

$$u$$

$$\dot{\eta }$$

$$u$$

$${\dot{\xi }}_i$$

$$u_i$$

$${\nu }_i$$

$${\dot{\xi }}_{i,\underline{i}}$$

$${\nu }_{i,\underline{i}}={\tilde{\nu }}_{i,\underline{i}}+E_{m_{\underline{i}}}^{\mathsf{T}}\sum _{k\in \overline{\stackrel{ˉ}{i}}}{E}_{m_{\underline{k}}}{\tilde{\nu }}_{k,\underline{k}}.$$

$$\begin{gathered} {\tilde{\nu }}_{i,\underline{i}}=E_{m_{\underline{i}}}^{\mathsf{T}}{F}_i\sum _{k\in \bar{i}}{\bar{S}}_n^{-1}(i,k){\xi }_k=F^i\sum _{k\in \bar{i}}{\bar{S}}_n^{-1}(i,k){\xi }_{k,\underline{i}} \\ =F^i\sum _{k\in \bar{i}}{\bar{S}}_n^{-1}(i,k)E_{n_{\underline{i}}}^{\mathsf{T}}{E}_{n_{\underline{k}}}{\xi }_{k,\underline{k}}. \end{gathered}$$

$$\mathcal{K}$$

$$\displaystyle\mathcal{P}={\left[\begin{array}[]{c|c}\bar{A}+\bar{B}\bar{F}+\tilde{L}\bar{C}&-\tilde{L}\bar{\mathbf{1}}_{p}\\ \hline\cr\rule{0.0pt}{11.19443pt}\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F}&0\end{array}\right]}.$$

$$\displaystyle\mathcal{P}_{2}\mathrel{\mathchoice{\vbox{\hbox{$\displaystyle:$}}}{\vbox{\hbox{$\textstyle:$}}}{\vbox{\hbox{$\scriptstyle:$}}}{\vbox{\hbox{$\scriptscriptstyle:$}}}{=}}{\left[\begin{array}[]{c|c}\bar{A}+\bar{B}\bar{F}+{L}_{x}\bar{C}&-\tilde{L}\bar{\mathbf{1}}_{p}\\ \hline\cr\rule{0.0pt}{11.19443pt}\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F}&0\end{array}\right]},$$

$$\mathcal{P}_{2}(s)=-\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F}\bigl{(}s\bar{I}_{n}-\bar{A}-\bar{B}\bar{F}-\bar{S}_{n}^{-1}\bar{L}\bar{S}_{p}\bar{C}\\ +\bar{S}_{n}^{-1}\bar{L}\bar{\mathbf{1}}_{p}\bar{\mathbf{1}}_{p}^{\mathsf{T}}\bar{C}-\bar{S}_{n}^{-1}\bar{L}\bar{\mathbf{1}}_{p}\operatorname{\mathrm{diag}}(\bar{\mathbf{1}}_{p}^{\mathsf{T}})\bar{C}\bigr{)}^{-1}\tilde{L}\bar{\mathbf{1}}_{p},$$

$$\mathcal{P}_{2}(s)=-\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F}\bigl{(}s\bar{I}_{n}-\bar{A}-\bar{B}\bar{F}-\bar{S}_{n}^{-1}\bar{L}\bar{S}_{p}\bar{C}\\ +\tilde{L}\bar{\mathbf{1}}_{p}\bar{\mathbf{1}}_{p}^{\mathsf{T}}\bar{C}-\tilde{L}\bar{\mathbf{1}}_{p}\operatorname{\mathrm{diag}}(\bar{\mathbf{1}}_{p}^{\mathsf{T}})\bar{C}\bigr{)}^{-1}\tilde{L}\bar{\mathbf{1}}_{p}.$$

$${\mathcal{P}}_2(s)=-{\bar{1}}_m^{\mathsf{T}}\bar{F}{\left(Z_1+Z_2\right)}^{-1}\tilde{L}{\bar{1}}_p.$$

$${\mathcal{P}}_2(s)=-{\bar{1}}_m^{\mathsf{T}}\bar{F}{Z}_1^{-1}\tilde{L}{\bar{1}}_p+M=\mathcal{P}(s)+\mathcal{M}(s),$$

$$\tilde{\mathcal{P}}\mathrel{\mathchoice{\vbox{\hbox{$\displaystyle:$}}}{\vbox{\hbox{$\textstyle:$}}}{\vbox{\hbox{$\scriptstyle:$}}}{\vbox{\hbox{$\scriptscriptstyle:$}}}{=}}\mathcal{P}-\operatorname{\mathrm{diag}}(\mathcal{P})=\left[\begin{array}[]{c|c}\bar{A}+\bar{B}\bar{F}+{L}_{x}\bar{C}&-\tilde{L}\bar{\mathbf{1}}_{p}\\ \hline\cr\rule{0.0pt}{11.19443pt}F_{\Delta}&0\end{array}\right],$$

Full text

††M. Kashyap and L. Lessard are with the Department of Electrical and Computer Engineering, at the University of Wisconsin–Madison, Madison, WI 53706, USA. L. Lessard is also with the Wisconsin Institute for Discovery at the University of Wisconsin–Madison. {mkashyap2,laurent.lessard}@wisc.edu

This material is based upon work supported by the National Science Foundation under Grant No. 1710892.

1 Introduction

Dynamically decoupled systems are examples of multi-agent systems where each agent has independent dynamics but the agents’ controllers share information via a communication network to achieve a common goal. Dynamically decoupled architectures occur naturally in the context of swarms of unmanned aerial vehicles (UAVs). For example, a swarm of UAVs might be deployed to survey an uncharted region or to optimize geographic coverage to combat a forest fire.

In this paper, we abstract the communication network as a directed acyclic graph (DAG). An example with five nodes is shown in Figure 1. Each plant $\mathcal{G}_{i}$ is assumed to obey state-space linear dynamics of the form

[TABLE]

where $x_{i}(t)\in\mathbb{R}^{n_{i}}$, $u_{i}(t)\in\mathbb{R}^{m_{i}}$, and $y_{i}(t)\in\mathbb{R}^{p_{i}}$ are the state, input, and measurement for each agent, and $w_{i}(t)\in\mathbb{R}^{q_{i}}$ is exogenous standard Gaussian noise, assumed to be independent across agents and time.

The global cooperative task is to optimize a standard infinite-horizon quadratic objective that couples the states and inputs of all agents. Specifically, if we form the global state by aggregating agents $\{1,\dots,N\}$ as x^{\mathsf{T}}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\begin{bmatrix}x_{1}^{\mathsf{T}}&\cdots&x_{N}^{\mathsf{T}}\end{bmatrix} and similarly for the aggregated $u$, $y$, and $w$, the cost function is of the form

[TABLE]

where the expectation $\operatorname{\mathbf{E}}$ is taken with respect to the exogenous noise $w$. If each controller $\mathcal{K}_{i}$ could measure the past history of all global measurements $y$, then finding the $\mathcal{K}_{i}$ that optimizes $\mathcal{J}$ in (2) would be a standard Linear Quadratic Gaussian (LQG) problem [20].

What makes our problem more difficult is that each $\mathcal{K}_{i}$ only has access to the measurement histories of its ancestor nodes in the DAG. In the example of Figure 1, $\mathcal{K}_{1}$ only measures $y_{1}$, $\mathcal{K}_{3}$ measures $\{y_{1},y_{3}\}$, $\mathcal{K}_{5}$ measures $\{y_{1},y_{4},y_{5}\}$, etc. All measurements are instantaneous.

Contributions.

We present an agent-level solution to the dynamically decoupled output-feedback problem. Specifically, we provide state-space realizations for the individual $\mathcal{K}_{i}$. Moreover, we provide intuitive interpretations for the controller states that highlight the role of the underlying graph structure and characterize the signals that should be transmitted between controllers. This work builds on [4, 3], which reported a state-space realization of the aggregated optimal controller but no agent-level details. Our results are presented in continuous time with an infinite-horizon cost but can easily be generalized to discrete time and/or a finite-horizon cost.

Literature review.

The elegant solutions found for optimal centralized control do not generally carry over to the decentralized case. It was shown for example that under LQG assumptions, linear compensators can be strictly suboptimal [18]. Several related decentralized control problems were shown to be intractable [1].

Tractability is closely related to how information is shared between the different controllers. For LQG problems with partially nested information, the optimal controller is linear [2]. When the control structure is quadratically invariant with respect to the plant, the problem of finding the optimal linear controller can be cast as a convex optimization over a set of stable Youla parameters [9, 10]. Under mild assumptions, convexity implies the existence of a unique optimal controller and efficient numerical methods for finding it. Numerical approaches include vectorization [11, 17] and Galerkin-style finite-dimensional approximation [12, 8].

The optimal decentralized controller often carries a rich structure, similar to the estimator-controller separation structure of classical centralized LQG control [19]. Such structure is lost in the aforementioned numerical approaches, which prompted recent efforts to solve these problems in an explicit way that uncovers the structure and intuition reminiscent of the centralized theory.

Explicit solutions for problems that are both partially nested and quadratically invariant have appeared for a variety of special cases. For example, state-feedback problems over general graphs where agent dynamics are coupled according to the graph structure [14, 5, 15]. Some output-feedback structures have also been solved, including two-player [7], broadcast [6], triangular [16], and dynamically decoupled [4, 3] cases. In the aforementioned works, explicit solutions typically consist of a state-space realization for the structured map $y\mapsto u$ between the global measurements $y$ and the global inputs $u$.

In this paper, we look at the dynamically decoupled case and drill down even further. We provide an agent-level solution, which is an explicit state-space realization of the optimal controllers for each of the agents that is both minimal and easily interpretable. Section 2 covers notation, formal assumptions, and other preliminaries. Section 3 presents our alternative form for the optimal controller, followed by interpretations of the controller states and signals in Section 4 and the explicit agent-level structure in Section 5. We conclude in Section 6.

2 Preliminaries

Notation.

A square matrix is Hurwitz if all eigenvalues have a strictly negative real part. $\mathcal{R}_{\textup{p}}$ is the set of real rational proper transfer matrices and $\mathcal{RH}_{\infty}$ is the set of real-rational proper transfer matrices with no poles on the imaginary axis and no unstable poles. Similarly, $\mathcal{RH}_{2}$ is the set of real-rational strictly proper transfer matrices with no poles on the imaginary axis and no unstable poles. We use the state-space notation

[TABLE]

If the realization above is stabilizable and detectable, then $\mathcal{G}\in\mathcal{RH}_{\infty}$ if and only if $A$ is Hurwitz. The $\mathcal{H}_{2}$ norm of $\mathcal{G}$ is $\lVert{\mathcal{G}}\rVert_{2}^{2}=\frac{1}{2\pi}\int_{-\infty}^{\infty}\operatorname{\mathrm{tr}}(\mathcal{G}^{*}(jw)\mathcal{G}(jw))\,\mathrm{d}w$. All transfer matrices are represented using calligraphic symbols.

We use $N$ to denote the total number of agents and [N]\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\{1,\dots,N\}. The global state dimension is n\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}n_{1}+\cdots+n_{N} and similarly for $m$ and $p$. $I_{k}$ will represent an identity matrix of size $k$. $\operatorname{\mathrm{blkdiag}}(\{X_{i}\})$ is the block-diagonal matrix formed by the blocks $\{X_{1},\dots,X_{n}\}$ and $\operatorname{\mathrm{diag}}(X)$ is the block-diagonal matrix formed by the diagonal blocks of $X$. The zeros used throughout are matrix or vector zeros and their sizes are determined from context. The symbol $\star$ represents a possible nonzero value in a matrix with dimensions inferred from context.

For $i\in[N]$, we write $\underline{i}\subseteq[N]$ to denote the descendants of node $i$, i.e., the set of nodes $j$ such that there is a directed path from $i$ to $j$. Likewise, we write $\bar{i}\subseteq[N]$ to denote the ancestors of node $i$. Similarly, $\bar{\bar{i}}$ and $\underline{\underline{i}}$ denote the strict ancestors and strict descendants, respectively. For example, in the graph of Figure 1, $\underline{1}=[5]$, $\underline{4}=\{4,5\}$, and $\bar{\bar{5}}=\{1,4\}$. We also use this notation as a means of indexing matrices. For example, if $X$ is a $N\times N$ block matrix, then we can write: $X_{2\underline{4}}=\begin{bmatrix}X_{24}&X_{25}\end{bmatrix}$.

Finally, we will require the use of specific partitions of the identity matrix. I_{n}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\operatorname{\mathrm{blkdiag}}(\{I_{n_{i}}\}) and for each agent $i\in[N]$, we define E_{n_{i}}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}(I_{n})_{:i} (the $i^{\textup{th}}$ block column of $I_{n}$). Similar to the descendant and ancestor definitions, $n_{\underline{i}}=\sum_{k\in\underline{i}}n_{k}$ and $n_{\bar{i}}=\sum_{k\in\bar{i}}n_{k}$. The dimensions of $E_{{n_{\bar{i}}}}$ and $E_{{n_{\underline{i}}}}$ are determined by the context of use. $1_{n}$ is the $n\times 1$ matrix of $1$’s. The definitions above also hold for other sets of indices, for example if we replace $n$ by $m$ or $p$.

Problem statement.

Aggregating (1) across agents, we can form global dynamics where $x$, $u$, $w$, and $y$ are the global state, input, exogenous noise, and measured output, respectively. We also define the regulated output $z=C_{1}x+D_{12}u$. The transfer matrix $(w,u)\mapsto(z,y)$ is

[TABLE]

The dynamically decoupled structure of (1) implies that A\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\operatorname{\mathrm{blkdiag}}(\{A_{ii}\}) and similarly for $B_{1}$, $B_{2}$, $C_{2}$, $D_{21}$. The cost matrices $C_{1}$ and $D_{12}$ are not assumed to have any special structure. It follows that $\mathcal{W}$ and $\mathcal{G}$ are block diagonal but $\mathcal{U}$ and $\mathcal{V}$ need not be.

The problem is to find the optimal controller $\mathcal{K}$ that solves the optimization problem

[TABLE]

We assume the DAG characterizing the information-sharing pattern has an adjacency matrix $S\in\{0,1\}^{N\times N}$, and $\mathbb{S}_{\mathcal{R}_{\textup{p}}}$ is the set of proper matrix transfer functions with a block-sparsity pattern corresponding to $S$. Namely, \mathbb{S}_{\mathcal{R}_{\textup{p}}}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\left\{\mathcal{K}\in\mathcal{R}_{\textup{p}}^{m\times p}\;|\;\mathcal{K}_{ij}=0\text{ if }S_{ij}=0\right\}. Note that the sub-blocks of $\mathcal{K}$ satisfy $\mathcal{K}_{ij}\in\mathcal{R}_{\textup{p}}^{m_{i}\times p_{j}}$. The optimal controller in this case is known to be linear and finite-dimensional [3]. So there is no loss of generality in assuming $\mathcal{K}\in\mathbb{S}_{\mathcal{R}_{\textup{p}}}$ in (3). In the sequel, we make several simplifying assumptions regarding the adjacency matrix and the system matrices.

Assumption 1** (Information sharing structure).**

Information is shared instantaneously between agent $i$ and its descendants $\underline{i}$, so there is an implicit edge from $i$ to each $j\in\underline{i}$ (the graph is transitively closed). Thus, any directed cycle can be treated as a single node and the graph becomes a DAG. By relabeling the nodes according to the paritial ordering induced by the DAG, the adjacency matrix $S\in\mathbb{R}^{N\times N}$ becomes lower triangular. $S$ is also invertible because each node has a self-loop.

Assumption 1 is critical in ensuring that the problem is partially nested [2] and quadratically invariant [10, 11], so the optimal controller is linear and finite-dimensional.

Riccati assumptions.

Matrices $(A,B,C,D)$ are said to satisfy the Riccati assumptions [3] if:

1. R1.

$C^{\mathsf{T}}D=0$ and $D^{\mathsf{T}}D>0$. 2. R2.

$(A,B)$ is stabilizable. 3. R3.

$\begin{bmatrix}A-j\omega I&B\\ C&D\end{bmatrix}$ is full column rank for all $\omega\in\mathbb{R}$.

If the Riccati assumptions hold, there is a unique stabilizing solution to the associated algebraic Riccati equation, which we write as $(X,F)=\operatorname{\mathrm{Ric}}(A,B,C,D)$. Thus $X\succ 0$ satisfies $A^{\mathsf{T}}X+XA+C^{\mathsf{T}}C-XB(D^{\mathsf{T}}D)^{-1}B^{\mathsf{T}}X=0$ with $A+BF$ Hurwitz, where F\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}-(D^{\mathsf{T}}D)^{-1}B^{\mathsf{T}}X.

Assumption 2** (System assumptions).**

For the $N$ interacting agents, we will assume the following.

1. 2.1.

As described in Section 2, the system is dynamically decoupled, A\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\operatorname{\mathrm{blkdiag}}(\{A_{ii}\}) and similarly for $B_{1}$, $B_{2}$, $C_{2}$, and $D_{21}$. The cost matrices $C_{1}$ and $D_{12}$ are not assumed to have any special structure. 2. 2.2.

$A_{ii}$* is Hurwitz for all $i\in[N]$, i.e., $A$ is Hurwitz.* 3. 2.3.

The Riccati assumptions hold for $(A,B_{2},C_{1},D_{12})$ and for $(A_{ii}^{\mathsf{T}},C_{2_{ii}}^{\mathsf{T}},B_{1_{ii}}^{\mathsf{T}},D_{{21}_{ii}}^{\mathsf{T}})$ for all $i\in[N]$.

Assumption 2.2 is an assumption of nominal stability, carried over from Kim and Lall [3]. Assumption 2.3 ensures the necessary condition that the centralized LQR problem and the individual agents’ estimation problems are non-degenerate.

3 Main Results

Kim and Lall [3] provide an explicit state-space formula for the controller that optimizes (3) under Assumptions 1–2. This will be the starting point for our work.

Lemma 3** (​​[3, Cor. 14]).**

Suppose Assumptions 1–2 hold. The solution to (3) is $\mathcal{K}=\mathcal{P}(I+\mathcal{G}\mathcal{P}-\operatorname{\mathrm{diag}}(\mathcal{G}\mathcal{P}))^{-1}$, where the $i^{\text{th}}$ column of $\mathcal{P}\in\mathbb{S}_{\mathcal{R}_{\textup{p}}}$ is given by

[TABLE]

and we define the control and estimation gains as

[TABLE]

Lemma 3 provides a state-space realization of the optimal controller. Our first main result is an even more explicit formula for the optimal controller $\mathcal{K}$ that highlights the duality between estimation and control. We will also see in Section 5 that this new formula is more readily implementable on an agent-level basis.

Theorem 4**.**

Suppose Assumptions 1–2 hold. A realization for the optimal controller that solves (3) is

[TABLE]

where we have defined the new symbols

[TABLE]

For all $i\in[N]$, with $F^{i}$ and ${L^{i}}^{\mathsf{T}}$ from (7), we also defined

[TABLE]

Proof. See Appendix A for a complete proof.

Remark 5**.**

The realization (10) of Theorem 4 is not minimal. Indeed, the gain matrices $L_{i}$ and $F_{i}$ have been padded with zeros so that they have compatible dimensions. We have $\bar{A}\in\mathbb{R}^{nN\times nN}$, where recall $N$ is the number of agents and n\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}n_{1}+\dots+n_{N} is the global aggregate state dimension of all agents. We use this non-minimal form to simplify algebraic manipulations; we present a generically minimal realization of (10) in Section 5.

Applying the state transformation $x\mapsto\bar{S}_{n}x$ to (10), we obtain a dual realization for the optimal controller.

Corollary 6**.**

Suppose Assumptions 1–2 hold. A realization for the optimal controller that solves (3) is

[TABLE]

where the notation is the same as in Theorem 4.

4 Interpreting the optimal controller

State interpretation.

Based on Theorem 4, a realization of the controller (calling the state $\xi$) is

[TABLE]

We call $\xi$ the state coordinates. Informally, $\xi_{i}$ can be interpreted as agent $i$’s best estimate of the global state $x$ given the available information $y_{\bar{i}}$. Likewise, a realization from Corollary 6 with state $\eta$ is

[TABLE]

We call $\eta$ the innovation coordinates. Informally, $\eta$ captures improvements in state estimates as information is aggregated along the DAG. A block-diagram of the global controller is shown in Figure 2.

Relationship to posets.

The work by Shah and Parrilo [13, 14] considers a different version of this decentralized control problem but obtains a similar optimal structure. The authors consider agents communicating on a DAG, except all agents use state feedback rather than output feedback, and $A$ is not required to be block-diagonal; it may have a block-sparsity pattern conforming to that of the DAG. The resulting controller for this problem has a controller-estimator structure very similar to the one in Figure 2. What the state-feedback coupled case and the output-feedback decoupled case share in common is the notion of separability [3]. The more general output-feedback case with dynamic coupling is not separable and the control and estimation gains are generally coupled in that case [7]. Nevertheless, the fact that a state-feedback and output-feedback variants of the problem share a similar optimal controller structure suggests that there may exist an even broader class of control problems sharing this interesting structure.

5 Agent-level implementation

In this section, we further refine the global controller structure of Figure 2 to obtain agent-level implementations of the optimal controller.

The realizations (14) and (15) describe the optimal controller dynamics in terms of the global state $\xi$ or $\eta$. Define the partial input \nu\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\bar{S}_{m}\bar{F}\bar{S}_{n}^{-1}\xi, and further isolate the dynamics of each individual agent by writing the $i^{\text{th}}$ block-component of (14) as

[TABLE]

Note that (16) can be implemented by agent $i$ because updates only require information from ancestors $\bar{i}$. Specifically, each $\nu_{i}$ is a function of $\xi_{\bar{i}}$ alone. Moreover, (16a) does not depend on the full $y$, but rather only $y_{\bar{i}}$, since L_{i}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}E_{n_{\bar{i}}}L^{\bar{i}}E_{p_{\bar{i}}}^{\mathsf{T}} (defined in Theorem 4).

We will now subdivide (16) to obtain agent-level update equations for each of the agents. It turns out agent $i$ only needs to store $\xi_{i,\underline{i}}$, which are the components of $\xi_{i}$ corresponding to the descendants of $i$.

By isolating these components, (16a) reduces to

[TABLE]

In summary, the optimal agent-level controller for agent $i$ maintains a state $\xi_{i,\underline{i}}$ and uses update equations (17) and (16b). Agent $i$ receives $\{\xi_{k,\underline{k}}\}_{k\in\bar{\bar{i}}}$ and $\{\tilde{\nu}_{k,\underline{k}}\}_{k\in\bar{\bar{i}}}$ from its strict ancestors $\bar{\bar{i}}$, and transmits $\xi_{i,\underline{i}}$ and $\tilde{\nu}_{i,\underline{i}}$ to its strict descendants $\underline{\underline{i}}$. A block diagram of the agent-level structure is shown in Figure 3.

A similar structure can be derived for the coordinate choice of Corollary 6, which we omit.

Minimal realization.

As pointed out in Remark 5 and above, the realizations obtained in Theorem 4 and Corollary 6 are not minimal. The agent-level implementation encapsulates that each node $i$ doesn’t transmit information about the $[N]\setminus\underline{i}$ nodes. These correspond to the unobservable modes that can be removed. Therefore, the reduced realizations are:

[TABLE]

where ${\bar{E}_{\underline{n}}}=\operatorname{\mathrm{blkdiag}}(\{E_{n_{\underline{i}}}\})$. In both realizations above, the total number of states is $\sum_{i=1}^{N}n_{\underline{i}}$ and the state transition matrix has block-sparsity conforming to $S$.

6 Conclusions

In this work, our starting point was the explicit solution to the output-feedback dynamically decoupled cooperative control problem found by Kim and Lall [3]. We obtained an equivalent and compact realization of the optimal controller, which enabled us to derive agent-level implementations for the optimal control laws and to characterize precisely what each agent needs to receive, compute, and transmit.

Appendix A Proof of Theorem 4

In order to manipulate the expression for $\mathcal{K}$ in Lemma 3, we begin by finding a realization for $\mathcal{P}$ by aggregating the individual $\mathcal{P}_{\underline{i}i}$ given in (6). For ease of exposition, we augment the states of each $\mathcal{P}_{\underline{i}i}$ by adding modes that are simultaneously uncontrollable and unobservable. Specifically, we augment each $A_{\underline{ii}}$ to $A$ and likewise with $B_{2_{\underline{ii}}}$ to $B_{2}$ and $C_{2_{i\underline{i}}}$ to $C_{2}$. For $i\in[N]$, we also augment the $F^{i}$ by zero-padding to obtain F_{i}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}E_{m_{\underline{i}}}F^{i}E_{n_{\underline{i}}}^{\mathsf{T}}. The estimation gains are similarly zero-padded to obtain \tilde{L}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\operatorname{\mathrm{blkdiag}}(\{E_{n_{i}}L^{i}E_{p_{i}}^{\mathsf{T}}\}). So finally, we obtain

[TABLE]

The input and output matrices, $-\tilde{L}\bar{\mathbf{1}}_{p}$ and $\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F}$ respectively, also arose by zero-padding so that all the modes we added to $\mathcal{P}$ are both uncontrollable and unobservable. Now consider the related transfer matrix

[TABLE]

where L_{x}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\bar{S}_{n}^{-1}\bar{L}(\bar{S}_{p}-\bar{\mathbf{1}}_{p}(\bar{\mathbf{1}}_{p}^{\mathsf{T}}-\operatorname{\mathrm{diag}}(\bar{\mathbf{1}}_{p}^{\mathsf{T}}))). We will now prove that $\mathcal{P}(s)=\mathcal{P}_{2}(s)$, which can be done by explicitly computing the transfer matrices and performing appropriate algebraic manipulations. Beginning with $\mathcal{P}_{2}$,

[TABLE]

where \bar{I}_{n}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}I_{N}\otimes I_{n}=I_{Nn}. By the sparsity pattern of $\tilde{L}$ and $\bar{S}_{n}$, we can verify that $\bar{S}_{n}\tilde{L}\bar{\mathbf{1}}_{p}=\bar{L}\bar{\mathbf{1}}_{p}$. Therefore,

[TABLE]

Now, $\tilde{L}\bar{\mathbf{1}}_{p}\operatorname{\mathrm{diag}}({\bar{\mathbf{1}}_{p}^{\mathsf{T}}})\bar{C}=\tilde{L}\bar{C}$. Defining Z_{1}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}(s\bar{I}_{n}-\bar{A}-\bar{B}\bar{F}-\tilde{L}\bar{C}) and Z_{2}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}-\bar{S}_{n}^{-1}\bar{L}\bar{S}_{p}\bar{C}+\tilde{L}\bar{\mathbf{1}}_{p}\bar{\mathbf{1}}_{p}^{\mathsf{T}}\bar{C},

[TABLE]

Applying the Woodbury matrix identity,

[TABLE]

where \mathcal{M}(s)\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F}Z_{1}^{-1}Z_{2}(\bar{I}_{n}+Z_{1}^{-1}Z_{2})^{-1}Z_{1}^{-1}\tilde{L}\bar{\mathbf{1}}_{p}. The matrix $Z_{1}^{-1}$ is block-diagonal, where each $Z_{1_{ii}}$ is also block-diagonal. Partitioning the rows and columns according to observable and unobservable modes, respectively, we obtain the following block-sparsity patterns: $Z_{1}^{-1}\sim\left(\begin{smallmatrix}\star&0\\ 0&\star\end{smallmatrix}\right)$, $Z_{2}\sim\left(\begin{smallmatrix}0&\star\\ 0&\star\end{smallmatrix}\right)$, $\tilde{L}\bar{\mathbf{1}}_{p}\sim\left(\begin{smallmatrix}\star\\ 0\end{smallmatrix}\right)$, and $\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F}\sim\left(\begin{smallmatrix}\star&0\end{smallmatrix}\right)$. Substituting into the definition of $\mathcal{M}$, we conclude that $\mathcal{M}(s)=0$ and so from (21), we have $\mathcal{P}_{2}(s)=\mathcal{P}(s)$.

Next, we substitute this expression for $\mathcal{P}_{2}$ into the expression for $\mathcal{K}$ in Lemma 3, one term at a time.

[TABLE]

where F_{\Delta}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F}-\operatorname{\mathrm{diag}}(\bar{\mathbf{1}}_{m}^{\mathsf{T}}\bar{F})=(\bar{\mathbf{1}}_{m}^{\mathsf{T}}-\operatorname{\mathrm{diag}}(\bar{\mathbf{1}}_{m}^{\mathsf{T}}))\bar{F}. We evaluate, $(I+\mathcal{G}\mathcal{P}-\operatorname{\mathrm{diag}}(\mathcal{G}\mathcal{P}))^{-1}=(I+\mathcal{G}\tilde{\mathcal{P}})^{-1}=$

[TABLE]

Finally, $\mathcal{K}=\mathcal{P}(I+\mathcal{G}\tilde{\mathcal{P}})^{-1}=$

[TABLE]

Applying the similarity transform with T_{1}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\left(\begin{smallmatrix}\bar{I}_{n}&0&\bar{I}_{n}\\ 0&I_{n}&0\\ 0&0&\bar{I}_{n}\end{smallmatrix}\right) reveals an uncontrollable mode, which we eliminate:

[TABLE]

Applying the similarity transform T_{2}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\left(\begin{smallmatrix}I_{n}&0\\ 0&\bar{S}_{n}^{-1}\end{smallmatrix}\right) and using the relationships $\tilde{L}\bar{\mathbf{1}}_{p}C_{2}=\tilde{L}\bar{C}\bar{\mathbf{1}}_{n}$, $\bar{S}_{n}\bar{A}=\bar{A}\bar{S}_{n}$, $\bar{S}_{n}\bar{B}=\bar{B}\bar{S}_{m}$, and $\bar{C}\bar{S}_{n}^{-1}=\bar{S}_{p}^{-1}\bar{C}$, we obtain

[TABLE]

Finally, we apply T_{3}\mathrel{\mathchoice{\vbox{\hbox{\displaystyle:}}}{\vbox{\hbox{\textstyle:}}}{\vbox{\hbox{\scriptstyle:}}}{\vbox{\hbox{\scriptscriptstyle:}}}{=}}\left(\begin{smallmatrix}I_{n}&(\bar{\mathbf{1}}_{n}^{\mathsf{T}}-\operatorname{\mathrm{diag}}(\bar{\mathbf{1}}_{n}^{\mathsf{T}}))\bar{S}_{n}^{-1}\\ 0&\bar{I}_{n}\end{smallmatrix}\right) and use the relationship $\bar{S}_{n}\tilde{L}\bar{\mathbf{1}}_{p}=\bar{L}\bar{\mathbf{1}}_{p}$, eliminate the uncontrollable states, and obtain a reduced realization of the controller:

[TABLE]