Model-Free Practical Cooperative Control for Diffusively Coupled Systems

TL;DR

This paper presents a data-driven, model-free control framework for diffusively coupled systems that guarantees convergence to a formation, using passivity theory and network optimization to find suitable gains.

Contribution

It introduces a novel, data-based method to determine control gains for formation control in passive systems, including strategies to reduce conservatism through iterative sampling.

Findings

Guaranteed convergence to desired formation within an epsilon neighborhood.

A data-driven method to compute control gains based on passivity and network optimization.

Reduction of conservatism via iterative experiments and passivity shortage estimation.

Abstract

In this paper, we develop a data-based controller design framework for diffusively coupled systems with guaranteed convergence to an $\epsilon$-neighborhood of the desired formation. The controller is comprised of a fixed controller with an adjustable gain on each edge. Via passivity theory and network optimization we not only prove that there exists a gain attaining the desired formation control goal, but we present a data-based method to find an upper bound on this gain. Furthermore, by allowing for additional experiments, the conservatism of the upper bound can be reduced via iterative sampling schemes. The introduced scheme is based on the assumption of passive systems, which we relax by discussing different methods for estimating the systems' passivity shortage, as well as applying transformations passivizing them. Finally, we illustrate the developed model-free cooperative control scheme with a case study.

Figures

Equations

$${\Sigma }_i:\{\begin{array}{l}{\dot{x}}_i=f_i(x_i,u_i)\\ y_i=h_i(x_i,u_i)\end{array},{\Pi }_e:\{\begin{array}{l}{\dot{\eta }}_e={\varphi }_e({\eta }_e,{\zeta }_e)\\ {\mu }_e={\psi }_e({\eta }_e,{\zeta }_e),\end{array}$$

$${\Sigma }_i:\{\begin{array}{l}{\dot{x}}_i=u_i\\ y_i=x_i\end{array},{\Pi }_e:\{\begin{array}{l}{\dot{\eta }}_e={\zeta }_e\\ {\mu }_e={\eta }_e\end{array}.$$

$$\dot{x}=-f(x)+g(x)u;y=h(x).$$

$$\underset{y,\zeta }{\min }$$

$$s.t.$$

$$\underset{y,\zeta }{\min }$$

$$s.t.$$

$$m_i\le {\mathrm{u}}_{i,1}({\mathrm{y}}_{i,1}-{\mathrm{y}}_{i,0})+\cdots +{\mathrm{u}}_{i,r}({\mathrm{y}}_{i,r}-{\mathrm{y}}_{i,r-1})+{\mathrm{u}}_i^{\star }({\mathrm{y}}_i^{\star }-{\mathrm{y}}_{i,r}).$$

$$m_i=K_i^{\star }({\mathrm{y}}_i^{\star })+K_i(0)$$

$$0\in k^{-1}(\mathrm{y}_{i})+\beta_{i}(\mathrm{y}_{i}-\mathrm{y}_{\rm ref})=\nabla\bigg{(}K^{\star}_{i}(\mathrm{y}_{i})+\frac{\beta_{i}}{2}(\mathrm{y}_{i}-\mathrm{y}_{\rm ref})^{2}\bigg{)},$$

$$m_i\le M_i\triangleq \sum _{k=1}^r{U}_{i,k}(Y_{i,k}-Y_{i,k-1})+U_{i,r+1}({\mathrm{y}}_i^{\star }-Y_{i,r}).$$

$${\int }_{Y_{i,k-1}}^{Y_{i,k}}|k^{-1}(s)-k^{-1}(Y_{i,k})|ds\le L{\int }_{Y_{i,k-1}}^{Y_{i,k}}|s-Y_{i,k}|ds=L{\Delta }_k^2.$$

$$L\left[\sum _{k=2}^r(Y_{i,k}-Y_{i,k-1}{)}^2+(y_i^{\star }-Y_{i,r}{)}^2+(Y_{i,1}-{\mathrm{y}}_{i,0}{)}^2\right]$$

$$\le$$

$$a_e^{(j+1)}=a_e^{(j)}+h{v}_e,e\in \mathbb{E},$$

$$v_e=\{\begin{array}{ll}\frac{{\mathrm{f}}_e-{\text{upzeta}}_e^{\star }}{{\gamma }_e({\mathrm{f}}_e)}& {\gamma }_e({\mathrm{f}}_e)\ne 0\\ 0& \text{otherwise}\end{array},e\in \mathbb{E}.$$

$${\Sigma }_i:\{\begin{array}{ll}\dot{x_i}& =-f_i(x_i)+u_i+w_i\\ y_i& =x_i,\end{array}$$

$$\frac{d}{dt}S(x)=\frac{h(x)-h(x_0)}{g(x)}(-f(x)+g(x)u)=$$

$$(h(x)-h(x_0))u-\frac{f(x)}{g(x)}(h(x)-h(x_0))=$$

$$\displaystyle(h(x)-h(x_{0}))\bigg{(}u-\frac{f(x_{0})}{g(x_{0})}\bigg{)}-\bigg{(}\frac{f(x)}{g(x)}-\frac{f(x_{0})}{g(x_{0})}\bigg{)}(h(x)-h(x_{0})),$$

$$\frac{f(x_0)}{g(x_0)}>\frac{f(x_1)}{g(x_1)}⟺x_0>x_1⟺h(x_0)>h(x_1)$$

$$\displaystyle\lim_{|t|\to\infty}\bigg{\|}\bigg{(}\frac{f(t)}{g(t)},h(t)\bigg{)}\bigg{\|}\geq\lim_{|t|\to\infty}\max\bigg{\{}\bigg{|}\frac{f(t)}{g(t)}\bigg{|},|h(t)|\bigg{\}}=\infty$$

$$X(\mathrm{y})=[\mathrm{diag}(\nabla k^{-1}(\mathrm{y}))+\mathcal{E}\mathrm{diag}(\nabla \gamma ({\mathcal{E}}^T\mathrm{y})){\mathcal{E}}^T{]}^{-1}.$$

$$\nabla F(a)=-\mathrm{diag}(\gamma ({\mathcal{E}}^T\mathrm{y}(a))){\mathcal{E}}^{T}X(\mathrm{y}(a))\mathcal{E}({\mathcal{E}}^T\mathrm{y}(a)-{\zeta }^{\star }).$$

$$v^T\nabla F(a)=-(\mathcal{E}({\mathcal{E}}^T\mathrm{y}(a)-{\zeta }^{\star }){)}^{T}X(\mathrm{y}(a))\mathcal{E}({\mathcal{E}}^T\mathrm{y}(a)-{\zeta }^{\star })$$

Full text

I Introduction

Multi-agent systems have received extensive attention in the past years, due to their appearance in many fields of engineering, exact sciences and social sciences. Examples include robotics[1], traffic engineering [2] and ecology [3]. The state-of-the-art approach to model-based control for multi-agent systems offers rigorous stability analysis, performance guarantees and systematic insights into the considered problem. However, with the growing complexity of systems, the modeling process is reaching its limits. Obtaining a reliable mathematical model of the agents becomes a time-intensive and arduous task.

At the same time, modern technology allows for gathering and storing more and more data from systems and processes, inciting an increasing interest in data-driven control. There are two main approaches for data-driven control. The first is model-based data-driven control, which uses data to identify a model from the problem, which is in turn used to solve the synthesis problem [4, 5]. In this case, the model estimation errors must be taken into account when solving the synthesis problem. The second is model-free control, which does not try to estimate a model for the system. The latter can be further bisected into approximate dynamic programming methods and direct policy search. The former evaluates a score for each state-action pair, and then obtains an optimal control policy using dynamic programming [6], and the latter tries to find the optimal policy directly, e.g. by gradient descent or via a non-parametric description of the possible trajectories [7]. These methods have all been applied to multi-agent systems as well, with varying degrees of success [8, 6, 9, 10].

In this work, we develop a data-driven controller synthesis approach for multi-agent systems that comes with rigorous theoretical analysis and stability guarantees for the closed loop, with almost no assumptions on the agents and few measurements needed. Our approach is based on high-gain control as well as passivity. Some ideas on high-gain approaches to cooperative control can be found in [11] and references therein. In [12], the authors provide a high-gain condition in the design of distributed $H_{\infty}$ controllers for platoons with undirected topologies, while there are also many approaches to (adaptively) tune the coupling weights, e.g. [13]. Our approach provides an upper bound on a high-gain controller using passivity measures. Passivity properties of the components can provide sufficient abstractions of their detailed dynamical models for guaranteed control. Such passivity properties can be obtained from data as ongoing work shows (e.g., [14, 15, 16]).

Passivity is a well-known tool for controller synthesis [17], which is useful, beyond convergence analysis, for its powerful properties such as compositionality. It was first introduced in the field of multi-agent systems in the seminal works of Arcak [18, 19], and was since explored in many variants in the context of multi-agent systems in many other works [20, 21, 22, 23, 24, 25, 26, 1]. We focus on the variant known as maximal equilibrium independent passivity (MEIP), presented in [22]. The notion of MEIP establishes a connection between multi-agent systems and network optimization, see [22, 23, 24, 25, 26]. Different synthesis problems have been solved using this network optimization framework assuming an exact model for each of the agents exists [23, 24, 27]. More precisely, one needs a perfect description of the steady-state input-output behavior of the agents. Thus, these methods cannot be applied in our case.

As we said, this work generally studies the problem of controller synthesis for diffusively coupled systems. The control objective is to converge to an $\epsilon$-neighborhood of a constant prescribed relative output vector. That is, for some tolerance $\epsilon>0$, we aim to design controllers so that the steady-state limit of the relative output is $\epsilon$-close to the prescribed values. The related problem of practical synchronization of multi-agent systems have been considered in [28], in which the agents were assumed to be known up to some bounded additive disturbance. However, a nominal model was needed to get practical synchronization. It was also pursued in [29], where strong coupling was used to drive agents close to a common trajectory, but again, a model for the agents was needed.

Our contributions are as follows. We present a model-free data-driven method for solving the practical formation control problem. This is done by cascading a fixed controller and an adjustable gain on each edge. We show that this gain can be chosen to guarantee a solution to the practical formation control problem. We then provide schemes to compute this gain offline only from input-output data without any model of the agents. In fact, this gain can be computed from only three experiments (per agent), and it can become less conservative with further data samples. If iterative experiments can be performed, we also provide an approach for applying different gains over different edges to further reduce any conservatism. We survey the different advantages for each of the methods and discuss their applicability in terms of the number of required measurements, or trade-offs in terms of energy. We also provide simulations presenting the effectiveness of the presented model-free control methods. To the best of the authors’ knowledge, no prior works consider data-driven control of multi-agent systems using passivity. Furthermore, this is the first application of the network optimization framework of [22, 24] where the agents do not have an exact model.

Notations

We employ use notions from algebraic graph theory [30]. An undirected graph $\mathcal{G}=(\mathbb{V},\mathbb{E})$ consists of finite sets of vertices $\mathbb{V}$ and edges $\mathbb{E}\subset\mathbb{V}\times\mathbb{V}$. We denote the edge having ends $i$ and $j$ in $\mathbb{V}$ by $k=\{i,j\}\in\mathbb{E}$. For each edge $k$, we pick an arbitrary orientation and denote $k=(i,j)$. The incidence matrix $\mathcal{E}\in\mathbb{R}^{|\mathbb{E}|\times|\mathbb{V}|}$ of $\mathcal{G}$ is defined such that for an edge $k=(i,j)\in\mathbb{E}$, we have $[\mathcal{E}]_{ik}=+1$, $[\mathcal{E}]_{jk}=-1$, and $[\mathcal{E}]_{\ell k}=0$ for $\ell\neq i,j$. $\mathrm{diam}(\mathcal{G})$ denotes the diameter of $\mathcal{G}$.

We also use notions from linear algebra. For every vector $v\in\mathbb{R}^{n}$, $\mathrm{diag}(v)$ denotes the $n\times n$ diagonal matrix with the entries of $v$ on its diagonal. The image of any linear map $T$ between vector spaces will be denoted by $\mathrm{Im}(T)$. Also, for two sets $A,B\subseteq\mathbb{R}^{d}$, we define $A+B=\{a+b:\ a\in A,\ b\in B\}$. Furthermore, $\|\cdot\|$ is the Euclidean norm.

Lastly, if $\Sigma$ is a dynamical system, and $M$ is a linear map of appropriate dimension, we can consider the cascaded system of $\Sigma$ and $M$. The cascade of $\Sigma$ after $M$ is denoted $\Sigma M$, and the cascade of $\Sigma$ before $M$ is denoted $M\Sigma$.

II Background: Network Optimization and

Passivity in Cooperative Control

Our goal in this subsection is to describe the diffusive coupling networks studied in [22], and to present the passivity-based network optimization framework achieved for multi-agent systems. See also [23, 24].

II-A Diffusively Coupled Systems and Steady-State Relations

Diffusively coupled networks are composed of agents $\{\Sigma_{i}\}_{i\in\mathbb{V}}$ interacting over a graph $\mathcal{G}=(\mathbb{V},\mathbb{E})$ using edge controllers $\{\Pi_{e}\}_{e\in\mathbb{E}}$. Each vertex $i\in\mathbb{V}$ represents an agent and each edge $e\in\mathbb{E}$ represents a controller. We model them as SISO dynamical systems:

[TABLE]

where the agents’ state, input and output are $x_{i},u_{i},y_{i}$ respectively, and the controllers’ state, input and output are $\eta_{e},\zeta_{e},\mu_{e}$ respectively. To understand the coupling of these systems, we consider the stacked inputs and outputs of the agents and controllers as $y=[y_{1},...,y_{|\mathbb{V}|}]^{T}$, and similarly for $u,\zeta,\mu$. The system is connected via the relations $\zeta=\mathcal{E}^{T}y$ and $u=-\mathcal{E}\mu$, where $\mathcal{E}$ is the incidence matrix of the graph $\mathcal{G}$. In other words, if we stack all agents to a dynamical system $\Sigma$, and stack all controllers to a dynamical system $\Pi$, the closed-loop is the feedback connection of $\Sigma$ and $\mathcal{E}\Pi\mathcal{E}^{T}$. See Fig. 1 for an illustration of the network, which we will denote by $(\mathcal{G},\Sigma,\Pi)$.

We will be interested in steady-states of the closed-loop system. It’s clear that if the stacked vectors $(\mathrm{u},\mathrm{y},\upzeta,\upmu)$ are a steady-state for $(\mathcal{G},\Sigma,\Pi)$, then for every vertex $i\in\mathbb{V}$, $(\mathrm{u}_{i},\mathrm{y}_{i})$ is a steady-state input-output pair for the system $\Sigma_{i}$, and for every edge $e\in\mathbb{E}$, $(\upzeta_{e},\upmu_{e})$ is a steady-state input-output pair for the system $\Pi_{e}$. This motivates the exploration of steady-state input-output relations, first defined in [22].

Definition 1**.**

The steady-state relation of a system is a set containing all the steady-state input-output pairs of the system.

We will denote the steady-states relations of $\Sigma_{i},\Pi_{e},\Sigma$, and $\Pi$ as $k_{i},\gamma_{e},k$, and $\gamma$, accordingly.

Remark 1**.**

We will sometimes abuse the notation and consider this relation as a set-valued map. Indeed, for any input $\mathrm{u}$ we can define the set $k(\mathrm{u})$ by $k(\mathrm{u})=\{\mathrm{y}:\ (\mathrm{u,y})\in k\},$ and similarly for $k_{i},\gamma_{e}$, and $\gamma$. We also consider the inverse relation $k^{-1}$ as the set-valued map assigning to a steady-state output $\mathrm{y}$ the set $k^{-1}(\mathrm{y})=\{\mathrm{u}:\ \mathrm{y}\in k(\mathrm{u})\},$ i.e., the set of all steady-state inputs corresponding to the steady-state output $\mathrm{y}$. We define this similarly for $k_{i},\gamma_{e}$, and $\gamma$.

Thus, $(\mathrm{u,y},\upzeta,\upmu)$ is a steady-state of $(\mathcal{G},\Sigma,\Pi)$ if and only if $\mathrm{y}\in k(\mathrm{u})$, $\upmu\in\gamma(\upzeta)$, $\upzeta=\mathcal{E}^{T}\mathrm{y}$ and $\mathrm{u}=-\mathcal{E}\upmu$. Equivalently, $\mathrm{y}$ is a steady-state output of $(\mathcal{G},\Sigma,\Pi)$ if and only if the zero vector [math] lies in the set $k^{-1}(\mathrm{y})+\mathcal{E}\gamma(\mathcal{E}^{T}\mathrm{y})$ [22, 23].

II-B Maximum Equilibrium-Independent Passivity and the Network Optimization Framework

The main tool allowing us to connect multi-agent systems to the network optimization world is monotone relations.

Definition 2**.**

A steady-state relation is monotone if for any two points $(\mathrm{u}_{1},\mathrm{y}_{1})$ and $(\mathrm{u}_{2},\mathrm{y}_{2})$ in the relation, $u_{1}<u_{2}$ implies $y_{1}\leq y_{2}$. We say that a monotone relation is maximally monotone if it is not contained in a larger monotone relation.

In order to connect this definition to the system-theoretic world, we define the following variant of passivity:

Definition 3** ([22]).**

A SISO system is said to be (output-strictly) maximum equilibrium-independent passive (MEIP) if the following two conditions hold:

• i)

The system is (output-strictly) passive with respect to any steady-state input-output pair it has, and

• ii)

it’s steady-state relation is maximally monotone.

One important property of maximally monotone relations is that they are subgradients of convex functions [31]. In this direction, we assume that the agents and controllers of the diffusively-coupled network $(\mathcal{G},\Sigma,\Pi)$ are MEIP. Let $K_{i}$, and $\Gamma_{e}$ be the corresponding convex integral functions for the steady-state relations $k_{i}$,$\gamma_{e}$. In other words, $\partial K_{i}=k_{i}$ and $\partial\Gamma_{e}=\gamma_{e}$, where $\partial$ denotes the subdifferential of convex functions [31]. We shall denote $K=\sum_{i\in\mathbb{V}}K_{i}$ and $\Gamma=\sum_{e\in\mathbb{E}}\Gamma_{e}$, so that $\partial K=k$ and $\partial\Gamma=\gamma$. The dual functions of $K_{i},\Gamma_{e},K,\Gamma$ are defined using the Legendre transform, $K^{\star}(\mathrm{y})=\sup_{\mathrm{u}}\{\mathrm{u}^{T}\mathrm{y}-K(\mathrm{u})\}=-\inf_{\mathrm{u}}\{K(\mathrm{u})-\mathrm{u}^{T}\mathrm{y}\}$, and similarly for $K_{i}^{\star},\Gamma_{e}^{\star}$ and $\Gamma^{\star}$. We note that $\partial K^{\star}=k^{-1}$, $\partial\Gamma^{\star}=\gamma^{-1}$, $\partial K_{i}^{\star}=k_{i}^{-1}$ and $\partial\Gamma_{e}^{\star}=\gamma_{e}^{-1}$ [31].

We now resume our interest in steady-states for the diffusively coupled network $(\mathcal{G},\Sigma,\Pi)$. We recall that $\mathrm{y}$ was the steady-state output of the diffusively coupled network if and only if $0\in k^{-1}(\mathrm{y})+\mathcal{E}\gamma(\mathcal{E}^{T}\mathrm{y})$. Restating this result in the language of convex functions gives the following theorem.

Theorem 1** ([22]).**

Consider the diffusively coupled network $(\mathcal{G},\Sigma,\Pi)$. Assume all agents $\Sigma_{i}$ are MEIP, and all controllers $\Pi_{e}$ are output-strictly MEIP (or vice versa). Let $k_{i},\gamma_{e},k$ and $\gamma$ be the steady-state relations of $\Sigma_{i},\Pi_{e},\Sigma$ and $\Pi$ accordingly, and let $K_{i},\Gamma_{e},K,\Gamma$ be the corresponding convex integral functions. Then the closed-loop system converges to a steady-state $(\mathrm{u,y},\upzeta,\upmu)$, such that $(\mathrm{y},\upzeta)$ and $(\mathrm{u},\upmu)$ are dual optimal solutions to the following convex optimization problems:

[TABLE]

The two network optimization problems above will be denoted often by (OPP) and (OFP) respectively. These problems are fundamental in the field of network optimization, dealing with optimization problems defined on graphs [31]. The names “optimal potential problem” and “optimal flow problem” are inspired from standard nomenclature in this field.

III Problem Formulation

We focus on relative-output based formation control. In this problem, the agents know the relative output $\upzeta_{e}=y_{i}-y_{j}$ with respect to their neighbors, and the control goal is to converge to a steady-state with prescribed relative outputs $\upzeta_{e}=\mathrm{y}_{i}-\mathrm{y}_{j}$. Examples include the consensus problem, in which all outputs must agree, as well as relative-position based formation control of robots, in which the robots are required to organize themselves in a desired spatial structure [32].

More specifically, we are given a graph $\mathcal{G}$ and agents $\Sigma$, and our goal is to design controllers $\Pi$ so that the signal $\zeta(t)$ of the diffusively coupled network $(\mathcal{G},\Sigma,\Pi)$ will converge to a desired, given steady-state vector $\upzeta^{\star}$. One evident solution is to apply a (shifted) integrator as a controller. However, this solution will not always work even when the agents are MEIP.

Example 1**.**

Consider agents $\Sigma_{i}$ with integrator dynamics, together with (shifted) integrator controllers $\Pi_{e}$, where we desire consensus (i.e., $\upzeta^{\star}=0$) over a connected graph $\mathcal{G}$,

[TABLE]

The trajectories of the diffusively-coupled system can be understood by noting that the closed-loop system yields the second-order dynamics $\ddot{x}=-\mathcal{E}\mathcal{E}^{T}x$. Decomposing $x$ using a basis of eigenvectors of the graph Laplacian $\mathcal{E}\mathcal{E}^{T}$, which is a positive semi-definite matrix, we see that the trajectory of $x(t)$ oscillates around the consensus manifold $\{\mathrm{x}:\ \exists\lambda\in\mathbb{R}\ \mathrm{x}=\lambda\mathbb{1}_{n}\}$. Specifically, $x(t)-\frac{1}{n}\mathbb{1}_{n}^{T}x(t)\mathbb{1}_{n}=\sum_{i=2}^{n}c_{i}\cos(\sqrt{\lambda_{i}}t+\varphi_{i})v_{i}$, where $\lambda_{2},\ldots,\lambda_{n}>0$ are the non-trivial eigenvalues of the graph Laplacian, $v_{2},\ldots,v_{n}$ are corresponding unit-length eigenvectors, and $c_{i},\varphi_{i}$ are constants depending on the initial conditions $x(0),\eta(0)$. Thus $x(t)=y(t)$ does not converge anywhere, let alone to consensus. Moreover, $\zeta(t)$ does not converge as $t\to\infty$, as $\mathcal{E}\zeta(t)=\mathcal{E}\mathcal{E}^{T}y(t)=\sum_{i=2}^{n}\lambda_{i}c_{i}\cos(\sqrt{\lambda_{i}}t+\varphi_{i})v_{i}$. Thus the integrator controller does not solve the formation control problem in this case.

Even if the integrator would solve this problem in general, we would like more freedom in choosing the controller. In practice, one might want to design the controller to satisfy extra requirements (like $\mathcal{H}_{2}$- or $\mathcal{H}_{\infty}$-norm minimization, or making sure that certain restrictions on the behavior of the system are not broken). We do not try and satisfy these more complex requirements, but instead show that a large class of controllers can be used to solve the practical formation control problem. In turn, this allows one to choose from a wide range of controllers, and try and satisfy additional desired properties. [23] offers an algorithm solving the problem, assuming the agents are MEIP and a perfect model of them is known. This algorithm allows a lot of freedom in the choice of controllers. However, in practice we oftentimes have no exact model of the agents, or any closed-form model. To formalize the goals we aim at, we define the notion of practical formation control.

Problem 1**.**

Given a graph $\mathcal{G}$, agents $\Sigma$, a desired formation $\upzeta^{\star}\in\mathrm{Im}(\mathcal{E}^{T})$, and an error margin $\varepsilon$, find a controller $\Pi$ so that the relative output vector $\zeta(t)$ of the network $(\mathcal{G},\Sigma,\Pi)$ converges to some $\upzeta_{0}$ such that $\|\zeta^{\star}-\upzeta_{0}\|\leq\varepsilon$.

By choosing suitable error margins $\varepsilon$, practical formation control (compared to formation control) comprises no restriction or real drawback in any application case. Therefore, solving the practical formation control problem constitutes an interesting problem especially for unknown dynamics of the agents. Thus, we strive to develop an algorithm solving this practical formation control problem without a model of the agents while still providing rigorous guarantees.

The underlying idea of our approach is amplifying the controller output. Consider the scenario depicted in Fig. 2, where the graph $\mathcal{G}$, the agents $\Sigma$ and the nominal controller $\Pi$ are fixed, and the gain matrix $A$ is a diagonal matrix $A=\mathrm{diag}(\{a_{e}\}_{e\in\mathbb{E}})$ with positive entries. We will show in the following that when the gains $a_{e}$ become large enough, the controller dynamics $\Pi$ become much more emphasized than the agent dynamics $\Sigma$. By correctly choosing the nominal controller $\Pi$ according to $\upzeta^{\star}$, we can hence achieve arbitrarily close formations to $\upzeta^{\star}$, as the effect of the agents on the closed-loop dynamics will be dampened. We denote the diffusively-coupled system in Fig. 2 as the 4-tuple $(\mathcal{G},\Sigma,\Pi,A)$, or as $(\mathcal{G},\Sigma,\Pi,a)$ where $a$ is the vector of diagonal entries of $A$. In case $A$ has uniform gains, i.e., $A=\alpha I$, we denote the system as $(\mathcal{G},\Sigma,\Pi,\alpha\mathbb{1}_{n})$ . We make an assumption in order to apply the network optimization framework of Theorem 1:

Assumption 1**.**

The agents $\{\Sigma_{i}\}_{i\in\mathbb{V}}$ are all MEIP, and the chosen controllers $\{\Pi_{e}\}_{e\in\mathbb{E}}$ are all output-strictly MEIP.

Before expanding on the suggested controller design, we discuss Assumption 1. In practice, we might not know if an agent is MEIP. Hence, we discuss how to either verify MEIP for the agents, or otherwise determine their shortage of passivity. We also discuss how to passivize the agents in the latter case. First, in some occasions, we might not know a model for the agents, but some known general structure properties. For example, one might know that an agent can be modeled by a gradient system, or a Euler-Lagrange system, but the exact model is unknown due to uncertainty on the characterizing parameters. In that case, we can use analytical results to verify MEIP. To exemplify this idea, we show how a very rough model can be used to prove that a system is MEIP.

Proposition 1**.**

Consider a control-affine SISO system:

[TABLE]

Assume that $g$ is positive, that $f/g,h:\mathbb{R}\to\mathbb{R}$ are continuous ascending functions, and that either $\lim_{|x|\to\infty}|f(x)/g(x)|=\infty$ or $\lim_{|x|\to\infty}|h(x)|=\infty$. Then (2) is MEIP.

The proof is available in the appendix. See also [24] for a treatment on gradient systems with oscillatory terms. Similarly, one can use a highly uncertain model to give an estimate about equilibrium-independent passivity indices.

Another approach for verifying Assumption 1 is learning input-output passivity properties from trajectories. For LTI systems, the shortage of passivity can be asymptotically revealed by iteratively probing the agents and measuring the output signal [15]. In [16], the authors showed that even one input-output trajectory (with persistently exciting input) is sufficient to find the shortage of passivity of an LTI system. For nonlinear agents, one can apply approaches presented in [14, 33], under an assumption on Lipschitz continuity of the steady-state relation. However, for general non-linear systems, this is still a work in progress. We note that for LTI systems, output-strict passivity directly implies output strict MEIP [22].

Using either approach, we can either find that an agent is MEIP, or that it has some shortage of passivity, and we need to render the agent passive in order to apply the model-free control approaches presented in this paper. We can use passivizing transformations in order to get a passive augmented agent. For example, if the agent has output-shortage of passivity $s_{i}>0$, we can apply a controller $C_{i}:y_{i}\mapsto\nu_{i}y_{i}$ to the agent as in [25], with $\nu_{i}>s_{i}$, as shown in Fig. 3. It can be shown that the augmented agent is output-strictly MEIP in this case. More generally, one could deal with more complex shortages of passivity, namely simultaneous input- and output-shortage of passivity, using more complex transformations [26].

With this discussion and relaxation of Assumption 1, we return to our solution of the practical formation control problem. We considered closed-loop systems of the form $(\mathcal{G},\Sigma,\Pi,a)$, where $a$ is a vector of edge gains. From here, the paper diverges into two sections. The next section deals with theory and analysis for uniform edge gains. The following section deals with the case of heterogeneous edge gains.

IV Practical Formation Control

with Uniform Gains

The chapter is split into two parts. The first part deals with the theory, and the second part deals with the corresponding implementation of practical formation control synthesis using uniform gains on the edges.

IV-A Theory

We wish to understand the effect of amplification on the steady-state of the closed-loop system. For the remainder of the section, we fix a graph $\mathcal{G}$, agents $\Sigma$ and controllers $\Pi$ such that Assumption 1 holds. We consider the diffusively coupled system $(\mathcal{G},\Sigma,\Pi,\alpha\mathbb{1}_{n})$ in Fig. 2, where the gains over all edges are identical and equal to $\alpha>0$, and wish to understand the effect of $\alpha$. We let $K$ and $\Gamma$ denote the sum of the integral functions of the agents and of the controllers, respectively. We first study the steady-states of this diffusively coupled system.

Lemma 1**.**

Under the assumptions above, the closed-loop converges to a steady-state, and the steady-states $\mathrm{y},\upzeta$ of the closed-loop system are minimizers of the following problem:

[TABLE]

Proof.

We define a new stacked controller, $\bar{\Pi}=\alpha\Pi$, by cascading the previous controller $\Pi$ with the gain $\alpha$. The resulting controller $\bar{\Pi}$ is again output-strictly MEIP, and we let $\bar{\gamma},\bar{\Gamma}$ denote the corresponding steady-state input-output relation and integral function. Theorem 1 implies that the closed-loop system (with $\bar{\Pi}$) converges to minimizers of (OPP) for the system $(\mathcal{G},\Sigma,\bar{\Pi})$. Hence, we have $\bar{\gamma}(\zeta)=\alpha\gamma(\zeta)$ for any $\zeta\in\mathbb{R}^{|\mathbb{E}|}$. Therefore, (OPP) for the system $(\mathcal{G},\Sigma,\bar{\Pi})$ reads:

[TABLE]

as $\bar{\Gamma}=\alpha\Gamma$. ∎

Our goal is to show that when $\alpha\gg 1$, the relative output vector $\zeta$ of the diffusively coupled system $(\mathcal{G},\Sigma,\Pi,\alpha\mathbb{1}_{n})$ globally asymptotically converges to an $\varepsilon=\varepsilon(\alpha)$-ball around the minimizer of $\Gamma$, and $\lim_{\alpha\to\infty}\varepsilon(\alpha)=0$. Thus, if we design the controllers so that $\Gamma$ is minimized at $\zeta^{\star}$, then $\alpha\gg 1$ provides a solution to the $\varepsilon$-practical formation control problem. Indeed, we can prove the following theorem:

Theorem 2**.**

Consider the closed-loop system $(\mathcal{G},\Sigma,\Pi,\alpha\mathbb{1}_{n})$, where the agents are MEIP and the controllers are output-strictly MEIP. Assume $\Gamma$ has a unique minimizer in $\mathrm{Im}(\mathcal{E}^{T})$, denoted $\zeta_{1}$. For any $\varepsilon>0$, there exists some $\alpha_{0}>0$, such that for all $\alpha>\alpha_{0}$ and for all initial conditions, the closed-loop converges to a vector $\mathrm{y}$ satisfying $\|\mathcal{E}^{T}\mathrm{y}-\zeta_{1}\|<\varepsilon$. In particular, if $\zeta_{1}=\zeta^{\star}$, we solve the practical control problem.

In order to prove the theorem, we study (OPP) for the diffusively coupled system $(\mathcal{G},\Sigma,\Pi,\alpha\mathbb{1}_{n})$, as described in Lemma 1. In order to do so, we need to prove a couple of lemmas. The first deals with lower bounds on the values of convex functions away from their minimizers.

Lemma 2**.**

Let $U$ be a finite-dimensional vector space. Let $f:U\to\mathbb{R}$ be convex, and suppose $x_{0}\in U$ is the unique minimum of $f$. Then for any $\delta>0$ there is some $M>f(x_{0})$ such that for any point $x\in U$, if $f(x)<M$ then $\|x-x_{0}\|<\delta$.

Proof.

We assume without loss of generality that $f(x_{0})=0$. Let $\mu$ be the minimum of $f$ on the set $\{x\in U:\ ||x-x_{0}||=\delta\}$, which is positive since $x_{0}$ is $f$’s unique minimum and the set $\{x\in U:\|x-x_{0}\|=\delta\}$ is compact. We know that, for any $y\in U$, the difference quotient $\frac{f(x_{0}+\lambda y)-f(x_{0})}{\lambda}$ is an increasing function of $\lambda>0$ (see Theorem 23.1 of [31]). Manipulating this inequality shows that for any $x\in U$, $||x||\geq\delta$ implies $f(x)\geq\frac{||x||}{\delta}\mu$, and in particular $f(x)\geq\mu$ whenever $||x||\geq\delta$. Thus, if $f(x)<\mu$ then we must have $||x-x_{0}||<\delta$, so we can choose $M=\mu$ and complete the proof. ∎

The second lemma deals with minimizers of perturbed versions of convex functions on graphs.

Lemma 3**.**

Fix a graph $\mathcal{G}=(\mathbb{V},\mathbb{E})$ and let $\mathcal{E}$ be its incidence matrix. Let $K:\mathbb{R}^{|\mathbb{V}|}\to\mathbb{R}$ be a convex function, and let $\Gamma:\mathbb{R}^{|\mathbb{E}|}\to\mathbb{R}$ be a convex function with a unique minimum $\zeta_{1}$ when restricted to the set $\mathrm{Im}(\mathcal{E}^{T})$. For any $\alpha>0$, consider the function $F_{\alpha}(\mathrm{y})=K^{\star}(\mathrm{y})+\alpha\Gamma(\mathcal{E}^{T}\mathrm{y})$. Then for any $\varepsilon>0$, there exists some $\alpha_{0}>0$ such that if $\alpha>\alpha_{0}$ then all of $F_{\alpha}$’s minima, $\mathrm{y}$, satisfy $\|\mathcal{E}^{T}\mathrm{y}-\zeta_{1}\|<\varepsilon$.

Proof.

By subtracting constants from $K{{}^{\star}}$ and $\Gamma$, we may assume without loss of generality that $\min(K{{}^{\star}})=\min(\Gamma)=0$. Choose some $\mathrm{y}_{0}\in\mathbb{R}^{|\mathbb{V}|}$ such that $\mathcal{E}^{T}\mathrm{y}_{0}=\zeta_{1}$ and let $m=K{{}^{\star}}(\mathrm{y}_{0})$. Note that $F_{\alpha}(y_{0})=m$, meaning that if $\mathrm{y}$ is any minimum of $F_{\alpha}$, it must satisfy $F_{\alpha}(\mathrm{y})\leq m$, and in particular $\Gamma(\mathcal{E}^{T}\mathrm{y})\leq\frac{m}{\alpha}$. Now, from Lemma 2 we know that there is some $M>0$ such that if $\Gamma(\mathcal{E}^{T}\mathrm{y})<M$ then $\|\mathcal{E}^{T}\mathrm{y}-\zeta_{1}\|<\varepsilon$. If we choose $\alpha_{0}=\frac{m}{M}$, then whenever $\alpha>\alpha_{0}$ we have $\Gamma(\mathcal{E}^{T}\mathrm{y})<M$, implying $\|\mathcal{E}^{T}\mathrm{y}-\zeta_{1}\|<\varepsilon$. ∎

We now connect the pieces and prove Theorem 2.

Proof.

Lemma 1 implies that the closed-loop system always converges to a minimizer of (3). Lemma 3 proves that there exists $\alpha_{0}>0$ such that if $\alpha>\alpha_{0}$ then all minimizers of (OPP) satisfy $\|\mathcal{E}^{T}\mathrm{y}-\zeta_{1}\|<\varepsilon$. This proves the theorem. ∎

Remark 2**.**

The parameters $\varepsilon$ and $\zeta^{\star}$ can be used to estimate the minimal required gain $\alpha_{0}$ by following the proofs of Lemma 2 and Lemma 3. Namely, $\alpha_{0}\leq\frac{m}{M}$ where $M$ is the minimum of $\Gamma$ on the set $\{\zeta\in\mathrm{Im}(\mathcal{E}^{T}):\ \|\zeta-\zeta^{\star}\|=\varepsilon\}$, and $m=K^{\star}(\mathrm{y}_{0})-\min_{y}{K^{\star}(\mathrm{y})}=K^{\star}(\mathrm{y}^{\star})+K(0)$ where $\mathrm{y}^{\star}\in\mathbb{R}^{|\mathbb{V}|}$ is any vector satisfying $\mathcal{E}^{T}\mathrm{y}^{\star}=\zeta^{\star}$.

Corollary 1**.**

Let $(\mathcal{G},\Sigma,\Pi,\alpha\mathbb{1}_{n})$ satisfy Assumption 1 and let $(\mathcal{G},\Sigma_{\mathrm{Int}},\Pi)$ be a network comprised of integrator dynamics for each agent. Denote the relative outputs of each system as $\zeta(t)$ and $\zeta_{\mathrm{Int}}(t)$ respectively. Then for any $\varepsilon>0$, there exists an $\alpha_{0}>0$ such that if $\alpha\geq\alpha_{0}$, then the relative outputs $\zeta(t)$ and $\zeta_{\mathrm{Int}}(t)$ both converge to constant vectors $\upzeta^{\star}$ and $\upzeta_{\mathrm{Int}}^{\star}$ respectively, and satisfy $\|\upzeta^{\star}-\upzeta_{\mathrm{Int}}^{\star}\|\leq\varepsilon$.

Proof.

The agents $\Sigma_{\mathrm{Int}}$ are MEIP. Thus, by Theorem 1, we know that the diffusively-coupled system $(\mathcal{G},\Sigma_{\mathrm{Int}},\Pi)$ converges to a steady-state, and its steady-state output is a minimizer of the associated (OPP) problem. Note that the input-output relation of $\Sigma_{\mathrm{Int}}$ is given via $k^{-1}(\mathrm{y})=0$, meaning the integral function $K^{\star}$ is the zero function. Thus the associated problem (OPP) is the unconstrained minimization of $\Gamma(\mathcal{E}^{T}\mathrm{y})$, meaning that the system $(\mathcal{G},\Sigma_{\mathrm{Int}},\Pi)$ converges, and its output converges to a minimizer of $\Gamma(\mathcal{E}^{T}\mathrm{y})$, i.e., its relative output $\zeta(t)$ converges to the minimizer of $\Gamma$ on $\mathrm{Im}(\mathcal{E}^{T})$. Applying Theorem 2 now completes the proof. ∎

Remark 3** (Almost Data-free control).**

Corollary 1 can be thought of as a synthesis procedure. Indeed, we can solve the synthesis problem as if the agents were single integrators, and then amplify the controller output by a factor $\alpha$. The corollary shows that for any $\varepsilon>0$, there is a threshold $\alpha_{0}>0$ such that if $\alpha>\alpha_{0}$, the closed-loop system converges to an $\varepsilon$-neighborhood of $\zeta^{\star}$. We note that we only know that $\alpha_{0}$ exists as long as the agents are MEIP. Computing an estimate on $\alpha_{0}$, however, requires one to conduct a few experiments.

There are a few possible approaches to overcome this requirement. One can try an iterative scheme, in which the edge gains are updated between iterations. Gradient-descent and extremum-seeking approaches are discussed in the next section (see Algorithm 3), but both require to measure the system between iterations. Another approach is to update the edge gains on a much slower time-scale than the dynamics of the system. This results in a two time-scale dynamical system, where the gains $a_{e}$ of the system $(\mathcal{G},\Sigma,\Pi,a)$ are updated slowly enough to allow the system to converge. Taking $a_{e}$ as uniform gains of size $\alpha$, and slowly increasing $\alpha$, assures that eventually, $\alpha>\alpha_{0}$, so the system will converge $\varepsilon$-close to $\zeta^{\star}$. The only data we need is whether or not the system has already converged to an $\varepsilon$-neighborhood of $\zeta^{\star}$111Such data can be obtained by different methods, e.g. checking the size of the derivatives, using physical intuition in some cases, or using passivity to determine convergence rates as in [27]. , to know whether $\alpha$ should be updated or not. This requires no data on the trajectories themselves, nor information on the specific steady-state limit, but only knowing whether the control goal has been achieved, which is the coarsest form of data. This results in an almost data-free solution of the practical formation control problem, which is valid as long as the agents are MEIP.

IV-B Data-Driven Determination of Gains

In the previous subsection, we introduced a formula for a uniform gain $\alpha$ described by the ratio of $m$ and $M$, that solves the practical formation problem, where $m$ and $M$ are as defined in Remark 2. The parameter $M$ depends on the integral function $\Gamma$ of the controllers, evaluated on well-defined points, namely $\{\zeta\in\mathrm{Im}(\mathcal{E}^{T}):\ \|\zeta-\upzeta^{\star}\|=\varepsilon\}$. Thus we can compute $M$ exactly with no prior knowledge on the agents. This is not the case for the parameter $m$, which depends on the integral function of the agents. Without knowledge of any model of the agents, we need to obtain an estimate of $m$ solely on the basis of input-output data from the agents.

From Remark 2, we know that $m=\sum_{i=1}^{n}(K_{i}^{\star}(\mathrm{y}_{i}^{\star})+K_{i}(0))=\sum_{i=1}^{n}m_{i}$ for some $\mathrm{y}^{\star}\in\mathbb{R}^{n}$ such that $\mathcal{E}^{T}\mathrm{y}^{\star}=\upzeta^{\star}$. Without a model of the agents, $m$ cannot be computed directly, but we can find an upper bound on $m$ from measured input-output trajectories via the inverse relations $k_{i}^{-1}$, $i=1,\dots,n$.

Proposition 2**.**

Let $(\mathrm{u}_{i}^{\star},\mathrm{y}_{i}^{\star})$, $(\mathrm{u}_{i,1}$, $\mathrm{y}_{i,1})$, $(\mathrm{u}_{i,2},\mathrm{y}_{i,2})$, $\dots$, $(\mathrm{u}_{i,r},\mathrm{y}_{i,r})$ and $(0,\mathrm{y}_{i,0})$ be steady-state input-output pairs for agent $i$, for some $r\geq 0$. Then:

[TABLE]

Proof.

We prove the claim by induction on the number of steady-state pairs, $r+2$. First, consider the case $r=0$ of two steady-state pairs. Because $(0,\mathrm{y}_{i,0})$ is a steady-state pair, we know that $K_{i}(0)=-K_{i}^{\star}(\mathrm{y}_{i,0})$ by Fenchel duality. Similarly, $K_{i}(\mathrm{u}_{i}^{\star})=\mathrm{u}_{i}^{\star}\mathrm{y}_{i}^{\star}-K_{i}^{\star}(\mathrm{y}_{i}^{\star})$. Thus,

[TABLE]

where we use the inequality $K^{\star}_{i}(b)-K^{\star}_{i}(c)\geq k^{-1}_{i}(c)(b-c)$ for $b=\mathrm{y}_{i,0}$ and $c=\mathrm{y}^{\star}_{i}$. Now, we move to the case $r\geq 1$. We write $m_{i}$ as $(K_{i}^{\star}(\mathrm{y}_{i}^{\star})-K_{i}^{\star}(\mathrm{y}_{i,r}))+(K_{i}^{\star}(\mathrm{y}_{i,r})-K_{i}(0))$. The first element can be shown to be bounded by $\mathrm{u}_{i}^{\star}(\mathrm{y}^{\star}_{i}-\mathrm{y}_{i,r})$ by the case $r=0$. The second element is bounded by $\mathrm{u}_{i,1}(\mathrm{y}_{i,1}-\mathrm{y}_{i,0})+\cdots+\mathrm{u}_{i,r}(\mathrm{y}_{i,r}-\mathrm{y}_{i,r-1})$ by induction hypothesis, as we use a total of $r+1$ steady-state pairs. Thus, $m_{i}$ is no greater than the sum of the two bounds, $\mathrm{u}_{i,1}(\mathrm{y}_{i,1}-\mathrm{y}_{i,0})+\cdots+\mathrm{u}_{i,r}(\mathrm{y}_{i,r}-\mathrm{y}_{i,r-1})+\mathrm{u}_{i}^{\star}(\mathrm{y}^{\star}_{i}-\mathrm{y}_{i,r})$. ∎

Remark 4**.**

If we only have two steady-state pairs, $(\mathrm{u}_{i}^{\star},\mathrm{y}_{i}^{\star})$ and $(0,\mathrm{y}_{i,0})$, the estimate on $m_{i}$ becomes $m_{i}\leq\mathrm{u}_{i}^{\star}(\mathrm{y}_{i}^{\star}-\mathrm{y}_{i,0})$. Thus two steady-state pairs, corresponding to two measurements/experiments, are enough to yield a meaningful bound on $m_{i}$. We do note that more experiments yield better estimates of $m_{i}$, i.e., if $r\geq 1$ the estimate in Proposition 2 is better as long as $(\mathrm{y}_{i,0},\mathrm{y}_{i,1},...,\mathrm{y}_{i,r},\mathrm{y}_{i}^{\star})$ is a monotone series.

With can use Remark 4 to compute an upper bound on $m$ from the two steady-state pairs $(\mathrm{u}_{i}^{\star},\mathrm{y}_{i}^{\star})$ and $(0,\mathrm{y}_{i,0})$ per agent. Designing experiments to measure these quantities is possible, but can require additional information on the plant, e.g. output-strict passivity. Instead, we take another path and estimate $\mathrm{y}_{i,0}$ and $\mathrm{u}_{i}^{\star}$ instead of computing them directly. Indeed, we use the monotonicity of the steady-state input-output relation to bound $\mathrm{u}_{i}^{\star}$ and $\mathrm{y}_{i,0}$ from above and below. The approach is described in Algorithm 1. It is important to note that the closed-loop experiments are done with a output-strictly MEIP controller, which assure that the closed-loop system indeed converges.

We prove the following:

Proposition 3**.**

The output $\mathbb{m}_{i}$ of Algorithm 1 is an upper bound on $m_{i}$.

Proof.

First, we show that the closed-loop experiments conducted by the algorithm indeed converge. The plant $\Sigma_{i}$ is assumed to be passive with respect to any steady-state input-output pair it possesses. Moreover, the static controller $u=\beta_{i}(y-y_{\rm ref})$ is output-strictly passive with respect to any steady-state input-output pair it possesses. Thus it is enough to show that the closed-loop system has a steady-state, which will prove convergence as this is a feedback connection of a passive system with an output-strictly passive system. Indeed, a steady-state input-output pair $(\mathrm{u_{i},y_{i}})$ of the system must satisfy $\mathrm{u}_{i}\in k_{i}^{-1}(\mathrm{y}_{i})$ and $\mathrm{u}_{i}=-\beta_{i}(\mathrm{y}_{i}-\mathrm{y}_{\rm ref})$, or $-\beta_{i}(\mathrm{y_{i}-y_{ref}})\in k^{-1}(\mathrm{y}_{i})$. This is equivalent to

[TABLE]

so $\mathrm{y}_{i}$ exists and is equal to the minimizer of $K^{\star}_{i}(\mathrm{y}_{i})+\frac{\beta_{i}}{2}(\mathrm{y}_{i}-\mathrm{y}_{\rm ref})^{2}$. This shows that the closed-loop experiments converge. it remains to show that it outputs an upper-bound on $m_{i}$.

Using Remark 4, it is enough to show that $\mathrm{y}_{i,0}\in[\underline{\mathrm{y}_{i,0}},\overline{\mathrm{y}_{i,0}}]$ and $\mathrm{u}_{i}^{\star}\in[\underline{\mathrm{u}_{i}^{\star}},\overline{\mathrm{u}_{i}^{\star}}]$. To do so, we first claim that $U_{1}\leq\mathrm{u}_{i}^{\star}\leq U_{3}$ and $Y_{1}\leq\mathrm{y}_{i,0}\leq Y_{3}$. We first show that $Y_{1}\leq\mathrm{y}_{i,0}$, by showing that $\mathrm{y}_{i,-}\leq\mathrm{y}_{i,0}$. Indeed, because $k_{i}$ is a monotone map, this is equivalent to saying that $\mathrm{u}_{i,-}\leq 0$. By the structure of the second experiment, the steady-state input is close to $-1$, and in particular smaller than [math]. The inequality $\mathrm{y}_{i,0}\leq\mathrm{y}_{i,+}$ is proved similarly. We note that because $\mathrm{u}_{i,-}\approx-1$ and $\mathrm{u}_{i,+}\approx 1$, we have $\mathrm{u}_{i,-}\leq\mathrm{u}_{i,+}$ and thus $\mathrm{y}_{i,-}\leq\mathrm{y}_{i,+}$. as $k_{i}$ is monotone.

Next, we prove that $U_{1}\leq\mathrm{u}_{i}^{\star}$. By monotonicity of $k_{i}$, this is equivalent to $Y_{1}\leq\mathrm{y}_{i}^{\star}$. Because $\mathrm{y}_{i,-}\leq\mathrm{y}_{i,+}$, it is enough to show that either $\mathrm{y}_{i,-}\leq\mathrm{y}_{i}^{\star}$ or $\mathrm{y}_{i,2}\leq\mathrm{y}_{i}^{\star}$. If the first case is true, then the proof is complete. Otherwise, $\mathrm{y}_{i,-}>\mathrm{y}_{i}^{\star}$, so the algorithm finds $\mathrm{y}_{i,2}$ by running the closed-loop system in Fig. 4 with $\beta_{i}=1$ and $\mathrm{y}_{\mathrm{ref}}\ll y_{i}^{\star}$. The increased coupling strength implies that the steady-state output $\mathrm{y}_{i,2}$ should be close to $\mathrm{y}_{\mathrm{ref}}$, which is much smaller than $\mathrm{y}_{i}^{\star}$. Thus $\mathrm{y}_{i,2}<\mathrm{y}_{i}^{\star}$, which shows that $Y_{1}\leq\mathrm{y}_{1}^{\star}$, or equivalently $U_{1}\leq\mathrm{u}_{1}^{\star}$. The proof that $\mathrm{u}_{1}^{\star}\leq U_{3}$ is similar. This completes the proof. ∎

Remark 5**.**

Algorithm 1 demands us to run a certain system with parameter $\beta_{i}$ small and wait until convergence. In practice, determining when the system has converged can be done by measuring the output or its derivative. Alternatively, one can use a Lypaunov-based approach [27]. One could also use engineering intuition and simulations to conclude an estimate on the termination time of the experiments. The parameter $\beta_{i}$ can be chosen in a similar manner.

Remark 6**.**

One can run more experiments to give tighter estimates of $m_{i}$. Indeed, by construction, take a collection $\{(U_{i,k},Y_{i,k})\}_{k=0}^{r+1}$ of steady-state input-output pairs, and use the monotonicity of the steady-state relation to sort them in such a way that $Y_{i,0}\leq\cdots\leq Y_{i,r}\leq y^{\star}_{i}\leq Y_{i,r+1}$, $U_{i,0}\leq 0\leq U_{i,1}\leq\cdots\leq U_{i,r+1}$. We would like to use Proposition 2, but we do not know the exact values of $y_{i,r},u_{i,0}$. Instead, we again use the monotonicity of the steady-state relation and bound $U_{i,r}\leq u_{i}^{\star}\leq U_{i,r+1}$ and $Y_{i,0}\leq y_{i,0}\leq Y_{i,1}$. Thus:

[TABLE]

We claim that $M_{i}$ is a relatively tight estimate of $m_{i}$. Indeed:

Proposition 4**.**

Let $\Delta_{k}=Y_{i,k}-Y_{i,k-1}\geq 0$, and assume that $k_{i}^{-1}$ is an $L$-Lipschitz function. Then $|M_{i}-m_{i}|\leq C\max\{L,1\}\max_{k}\Delta_{k}$, for a constant $C=C(\mathrm{y}_{i,0},\mathrm{u}_{i}^{\star},\mathrm{y}_{i}^{\star})$.

Proof.

First, $m_{i}=K_{i}^{\star}(\mathrm{y}_{i}^{\star})-K_{i}^{\star}(\mathrm{y}_{i,0})=\int_{\mathrm{y}_{i,0}}^{\mathrm{y}_{i}^{\star}}k_{i}^{-1}(s)ds$. Thus, it is enough to bound each of the following terms:

1. i)

$|\int_{Y_{i,k-1}}^{Y_{i,k}}k^{-1}_{i}(s)ds-U_{i,k}(Y_{i,k}-Y_{i,k-1})|,k=2,\cdots,r$. 2. ii)

$|\int_{Y_{i,r}}^{\mathrm{y}^{\star}_{i}}k^{-1}_{i}(s)ds-u_{i}^{\star}(\mathrm{y}^{\star}_{i}-Y_{i,r})|$. 3. iii)

$|\int_{\mathrm{y}_{i,0}}^{Y_{i,1}}k^{-1}_{i}(s)ds-U_{i,1}(Y_{i,1}-\mathrm{y}_{i,0})|$. 4. iv)

$|(U_{i,r+1}-u_{i}^{\star})(\mathrm{y}^{\star}_{i}-Y_{i,r})|$. 5. v)

$|U_{i,r+1}(Y_{i,r+1}-\mathrm{y}^{\star}_{i})|$ 6. vi)

$|U_{i,1}(\mathrm{y}_{i,0}-Y_{i,0})|$

The first term can be bounded by:

[TABLE]

Similarly, the second term is bounded by $L(y_{i}^{\star}-Y_{i,r})^{2}$ and the third term is bounded by $L(Y_{i,1}-\mathrm{y}_{i,0})^{2}$. The sum of the three terms is thus bounded by:

[TABLE]

where we use $y_{i}^{\star}-Y_{i,r}\leq\Delta_{r+1}$ and $Y_{i,1}-\mathrm{y}_{i,0}\leq\Delta_{1}$. Similarly, the fourth term is bounded by $L\Delta_{r+1}^{2}$, the fifth term is bounded by $U_{i,r+1}\Delta_{r+1}$ and the last term is bounded by $U_{i,1}\Delta_{0}$. This completes the proof, as $0\leq U_{i,1}\leq u_{i}^{\star}=k^{-1}_{i}(\mathrm{y}_{i}^{\star})$, and $U_{i,r+1}=k^{-1}(Y_{i,r+1})\leq k^{-1}(\mathrm{y}_{i}^{\star})+L\Delta_{r+1}$. ∎

Remark 7**.**

A natural question that arises is how to conduct experiments assuring that $\max_{k}\Delta_{k}$ is small. If we run the system in Fig. 4 with $\beta_{i}\gg 1$, then the steady-state output would be very close to $y_{\rm ref}$. Thus, if we run $r$ additional experiments with $\beta_{i}\gg 1$ and references $y_{\rm ref,1}\leq y_{\rm ref,2}\leq\cdots\leq y_{\rm ref,r}$ (i.e., a total of $r+3$ experiments), then $\max\Delta_{k}=O(\max_{k}|y_{\rm ref,k+1}-y_{\rm ref,k}|)$. Thus, choosing $y_{\rm ref,k}$ as $r$ equally spaced points in an appropriate interval would give $\max_{k}\Delta_{k}=O(1/r)$, and a uniformly random choice gives $\max_{k}\Delta_{k}=O(\log r/r)$ with high probability [34].

We saw that $m_{i}$ can be bounded using three experiments for general MEIP agents, and that additional measurements can be used to provide more accurate estimates of $m_{i}$. Other recent works about data-driven control focus on the case of LTI systems, using them as a base to build toward a solution for nonlinear systems [4, 7, 35]. If we restrict ourselves to this case, we can exactly compute $m_{i}$ from a single experiment:

Proposition 5**.**

Suppose that the agent $\Sigma_{i}$ is known to be both MEIP and LTI. Let $(\mathrm{\tilde{u},\tilde{y}})$ be any steady-state input-output pair for which either $\tilde{\mathrm{u}}\neq 0$ or $\tilde{\mathrm{y}}=0.$222e.g., by running the system in Fig. 4 with some $\beta>0$ and $\mathrm{y}_{\mathrm{ref}}\neq 0$ Then $m_{i}=\frac{\mathrm{(}\mathrm{y}_{i}^{\star})^{2}\mathrm{\tilde{u}}}{2\mathrm{\tilde{y}}}$. Thus $m_{i}$ can be exactly calculated using a single experiment.

Proof.

We know that $k$ is a linear function, and the system state matrix is Hurwitz [21, 24]. Moreover, unless the transfer function of the agent is [math], $k^{-1}$ is a linear function $k^{-1}(\mathrm{y})=s\mathrm{y}$ for some $s>0$ [36]. Thus $K^{\star}(\mathrm{y})=\frac{s}{2}\mathrm{y}^{2}$. Now, $k^{-1}(0)=s\cdot 0=0$, so $(0,0)$ is a steady-state input-output pair, meaning that $\mathrm{y}_{i,0}=0$. Moreover, we know that $\mathrm{\tilde{u}}=s\mathrm{\tilde{y}}$, and not both are zero, so we conclude that $\mathrm{\tilde{y}}\neq 0$, and that $s=\frac{\mathrm{\tilde{u}}}{\mathrm{\tilde{y}}}$. Thus, $K_{i}^{\star}(\mathrm{y}_{i,0})=K_{i}^{\star}(0)=0$ and $K_{i}^{\star}(\mathrm{y}_{i}^{\star})=\frac{s}{2}(\mathrm{y}_{i}^{\star})^{2}$. This completes the proof, as $m_{i}=K_{i}^{\star}(\mathrm{y}_{i}^{\star})-K_{i}^{\star}(\mathrm{y}_{i,0})$. ∎

The chapter concludes with Algorithm 2 for solving the practical formation control problem using the single-gain amplification scheme, which is applied in Section VI.

Remark 8**.**

Step 1 of the algorithm allows almost complete freedom of choice for the controllers. One possible choice are the static controllers $\mu_{e}=\zeta_{e}-\upzeta_{e}^{\star}$. Moreover, if $\Pi_{e}$ is any MEIP controller for each $e\in\mathbb{E}$, and $\gamma_{e}(\zeta_{e})=0$ has a unique solution for each $e\in\mathbb{E}$, then the “formation reconfiguration” scheme from [23] suggests a way to find the required controllers using mild augmentation.

Remark 9**.**

The algorithm allows one to choose any vector $\mathrm{y}^{\star}$ such that $\mathcal{E}^{T}\mathrm{y}^{\star}=\upzeta^{\star}$. All possible choices lead to some gain $\alpha$ which assures a solution of the practical formation control problem, but some choices yield better results (i.e., smaller gains) than others. The optimal $\mathrm{y}^{\star}$, minimizing the estimate $\alpha$, can be found as the minimizer of the problem $\min\{K^{\star}(\mathrm{y}):\ \mathcal{E}^{T}\mathrm{y}=\upzeta^{\star}\}$, which we cannot compute using data alone. One can use physical intuition to choose a vector $\mathrm{y}^{\star}$ which is relatively close to the actual minimizer, but the algorithm is still valid no matter which $\mathrm{y}^{\star}$ is chosen.

V Iterative Practical Formation Control: Applying Different Gains on Different Edges

Let us revisit Fig. 2 and let $A=\mathrm{diag}(\{a_{e}\}_{e\in\mathbb{E}})$ with positive, but distinct entries $a_{e}$. These additional degrees of freedom can be used, for example, to reduce the conservatism and retrieve a smaller norm of the adjustable gain vector $a$ while still solving the practical formation control problem. It follows directly from Theorem 2 that there always exists a bounded vector $a$ solving the practical formation control problem. However, the question remains how $a$ can be chosen based on sampled input-output data and passivity properties.

Our idea here is to probe our diffusively coupled system for given gains $a_{e}$ and adjust the gains according to the resulting steady-state output. By iteratively performing experiments in this way, we strive to find controller gains that solve the practical formation control problem. This approach is tightly connected to iterative learning control, where one iteratively applies and adjusts a controller to improve the performance of the closed-loop for a repetitive task [37]. Our approach here is based on passivity and network optimization with only requiring the possibility to perform iterative experiments.

One natural idea in this direction is to define a cost function that penalizes the distance of the resulting steady-state to the desired formation control goal and then apply a gradient descent approach, adjusting the gain $a$ for each experiment. However, to obtain the gradient of $\|\mathcal{E}^{T}\mathrm{y}(a)-\upzeta^{\star}\|^{2}$ with respect to the vector $a$, where $\mathrm{y}(a)$ is the steady-state output of $(\mathcal{G},\Sigma,\Pi,a)$, one requires knowledge of the inverse relations $k_{i}^{-1}$ for all $i=1,\dots,n$. With no model of the agents available, a direct gradient descent approach is hence infeasible. We thus look for a simple iterative multi-gain control scheme without knowledge on the exact steepest descent direction.

We start off with an arbitrarily chosen gain vector $a_{0}$ with only positive entries. Due to Assumption 1, the closed-loop converges to a steady state. According to the measured state, the idea is then to iteratively perform experiments and update the gain vector until we reach our control goal, i.e., practical formation control. The update formula can be summarized by

[TABLE]

where $h>0$ is the step size and $v$, with entries $v_{e}$, $e=1,\dots,|\mathbb{E}|$, is the update direction. In practice, the update can either be instantaneous or gradual, e.g. using linear interpolation or higher-order splines. We denote the $e$-th entry of $\mathcal{E}^{T}\mathrm{y}$ as $\mathrm{f}_{e}$ and choose $v$ in each iteration such that

[TABLE]

If $k^{-1}$ and $\gamma$ are differentiable functions, then we claim that $F(a)=||\mathcal{E}^{T}\mathrm{y}(a)-\upzeta^{\star}||^{2}$ decreases in the direction of $v$, i.e., $v^{T}\nabla F(a)<0$. This leads to a multi-gain distributed control scheme, using (5) with (6), summarized in Algorithm 3. This multi-gain distributed control scheme is guaranteed to solve the practical formation problem after a finite number of iterations, and is summarized in the following theorem.

Theorem 3**.**

Suppose that the functions $k^{-1}$, $\gamma$ are differentiable, and that there exists an agent $i_{0}\in\mathbb{V}$ such that $\frac{dk^{-1}_{i_{0}}}{d\mathrm{y}_{i_{0}}}>0$ for any point $\mathrm{y}_{i_{0}}\in\mathbb{R}$. Moreover, assume that $\frac{d\gamma_{e}}{d\mathrm{\zeta}_{e}}>0$ for any $e\in\mathbb{E},\zeta_{e}\in\mathbb{R}$. Then $v^{T}\nabla F(a)\leq 0,$ with $v,F$ as defined in Algorithm 3 (and equality if and only if $\mathcal{E}^{T}\mathrm{y}(a)=\upzeta^{\star}$). Furthermore, if the step size $h>0$ is small enough, then the Algorithm 3 halts after finite time, providing a gain vector that solves the practical formation control problem.

Sketch of proof.

The proof is based on showing that $\nabla F$ can be written as $-\mathrm{diag}(\gamma(\mathrm{f}))X(\mathrm{y}(a))(\mathrm{f}-\upzeta^{\star})$, where $X(\mathrm{y}(a))$ is a positive-definite matrix depending on $\mathrm{y}(a)$. We can now show that $v^{T}\nabla F=-(\mathrm{f}-\upzeta^{\star})^{T}X(\mathrm{y}(a))(\mathrm{f}-\upzeta^{\star})\leq 0$. The full proof of Theorem 3 is available in the appendix. ∎

Algorithm 3 together with the theoretical results from Theorem 3 provide us with a very simple and distributed, iterative control scheme with theoretical guarantees. Note also, that the steady-states of the agents are independent of their initial condition. For each iteration, the agents can hence also start from the position they converged to at the last iteration. This can be interpreted similarly to Remark 3, where gains are updated on a slower time scale than convergence of the agents. However, instead of only the information whether practical formation control is achieved, we generally need the actual difference $\mathcal{E}^{T}\mathrm{y}-\upzeta^{\star}$ that is achieved with the current controller in each iteration. In the special case of proportional controllers $\mu_{e}=\zeta_{e}-(\upzeta^{\star})_{e}$, yielding $v_{e}=1$, we retrieve the exact controller scheme proposed in Remark 3.

An alternative gradient-free control scheme is the extremum seeking framework presented in [38]. Assuming that $k^{-1}$ and $\gamma$ are twice continuously differentiable, a step in the direction of steepest descent is approximated every $4|\mathbb{E}|$ steps (cf. [38, Theorem 1]). While the extremum seeking framework approximates the steepest descent (and the simple multi-gain approach only guarantees a descending direction), it also requires large amounts of experiments per approximated gradient step. Furthermore, the algorithm as presented in [38] cannot be computed in a purely distributed fashion. Therefore, the simple distributed control scheme in Algorithm 3 displays significant advantages in the present problem setup.

VI Case Study: Velocity Coordination in Vehicles with Drag and Exogenous Forces

Consider a collection of $30$ one-dimensional robot vehicles, each modeled by a double integrator $G(s)=\frac{1}{s^{2}}$. The robots try to coordinate their velocity. Each of them has its own drag profile $f(\dot{p})$, which is unknown to the algorithm, but it is known that $f$ is increasing and $f(0)=0$. Moreover, each vehicle experiences external forces (e.g., wind, and being placed on a slope). The velocity of the vehicles is governed by the equation

[TABLE]

where $x_{i}$ is the velocity of the $i$-th vehicle, $f_{i}$ is its drag model, $w_{i}$ are the exogenous forces acting on it, $u_{i}$ is the control input, and $y_{i}$ is the measurement. In the simulation, the drag models $f_{i}$ are given by $c_{d}|x|x$, where the drag coefficient $c_{d}$ is chosen as a log-uniformly distributed random variable. We assume that the vehicles are light, so the wind accelerates the vehicles by a non-negligible amount. Thus, $w_{i}$ is randomly chosen between $-2$ and $2$. We wish to achieve velocity consensus, with error no greater than $\epsilon=0.2$. We consider a diffusive coupling of the agents with the cycle graph $\mathcal{G}=\mathcal{C}_{30}$, and take proportional controllers $\zeta_{e}=\mu_{e}$.

We apply the amplification scheme presented in Algorithm 2 and choose the consensus value $y^{\star}_{i}=1.5_{m/sec}$ to use in the estimation algorithm. Note that the plants are MEIP, but not output-strictly MEIP, and use Algorithm 1 to estimate the required uniform gain $\alpha$. The first two experiments are conducted with $\beta_{i}=0.01$, and $y_{\mathrm{ref}}=\pm 100$. Based on their results, we run a third experiment on each of the agents for which this is required, this time with $\beta_{i}=1$ and $y_{\mathrm{ref}}=\pm 10$, where the sign is chosen according to Algorithm 1. The experimental results are available in Figure 5(a).

We estimate each $m_{i}$ using Remark 4. For example, for agent 1 we get the three steady-state input-output pairs $(0.9947,0.5203)$, $(-0.9687,-3.1294)$, and $(3.4268,3.5732)$. Monotonicity implies that it has steady-states $(u^{\star}_{1},y^{\star}_{1})=(u^{\star}_{1},1.5)$ and $(0,y_{1,0})$ with $0.9947\leq u^{\star}_{1}\leq 3.4268$ and $-3.1294\leq y_{1,0}\leq 0.5203$. We can thus estimate $m_{2}\leq 3.4268\cdot(1.5-(-3.1294))=15.864$. Repeating this calculation for each of the agents and summing gives $m=256.3658$.

As for estimating $M$, we have $\Gamma_{e}(\zeta_{e})=\zeta^{2}_{e}$, so $\Gamma(\zeta)=\|\zeta\|^{2}$. The minimum is at $\zeta^{\star}=0$, and by definition we have $M=\min_{\zeta\in\mathrm{Im}(\mathcal{E}^{T}):\ \|\zeta-\upzeta^{\star}\|=\varepsilon}\Gamma(\zeta)=|\mathbb{E}|\varepsilon^{2}$. Thus, we get $\alpha=\frac{m}{M}=\frac{m}{30\varepsilon^{2}}=213.64$. To verify the algorithm, we run the closed-loop system $(\mathcal{G},\Sigma,\Pi,\alpha\mathbb{1})$ with the gain $\alpha$ we found. The results are available in Figure 5(b). One can see that the overall control goal is achieved - the agents converge to a steady-state which is $\varepsilon$-close to consensus. However, the agents actually converge to a much closer steady-state than predicted by the algorithm. Namely, the distance of the steady-state output from consensus is roughly $0.04$, much smaller than $0.2$. Uncoincidentally, the true value of $m=K(0)+K^{\star}(y^{\star})$ is $50.15$, meaning we overestimate it (and hence $m/M$) by about $411\%$. One can mitigate this by using more experiments to improve the estimate $m_{i}$, as in Proposition 2 or in Remark 6. We follow this approach and conduct further experiments on each of the agents using $\beta_{i}=10$ and choosing $y_{\mathrm{ref}}$ randomly. The resulting values of $\alpha$, as well as the error from the true value of $m/M$, can be seen in the table below. It can be seen that even a single additional measurement per agent can significantly reduce the estimation error of $m/M$.

[TABLE]

Altogether we showed that Algorithm 2 manages to solve the practical consensus problem for vehicles, affected by drag and exogenous inputs, without using any model for the agents, while conducting very few experiments for each agent. However, it overestimates the required coupling, and thus has unnecessarily large energy consumption.

Let us now apply the iterative multi-gain control strategy. We start with $a^{(0)}=0.1\otimes 1_{|\mathbb{E}|}$, we choose the step size $h=2$ and apply Algorithm 3. In fact, since $\upzeta^{\star}=0$ and $\zeta_{e}=\mu_{e}$, we receive $v=1_{|\mathbb{E}|}$, which constitutes the special case of Remark 3. The corresponding norm of the gain vector and the resulting $\varepsilon$ in each iteration is illustrated in Fig. 6. After $20$ iterations, we already arrive at a vector, which solves the practical formation problem with $\|a^{(19)}\|=208.68$, while $\varepsilon=0.195<0.2$. Note that the controller with the uniform gain had $\|a\|=\sqrt{|\mathbb{E}|}\cdot 213.638=1170.1$, so the iterative scheme beats it by a factor of $5$ in terms of energy.

VII Conclusions and Future Work

We presented an approach for model-free practical cooperative control for diffusively coupled systems only on the premise of passivity of the agents. The presented approach led to two control schemes: with additional two or three experiments on the agents, we can upper bound the controller gain which solves the practical formation problem, or we can iteratively adapt the adjustable gain vector until practical formation is reached. Both approaches are especially simple in their application, while still being scalable and providing theoretical guarantees. Future research might try and improve the presented methods, either by reducing the number of experiments needed on each agent, or by achieving faster practical convergence using iterations. One might also try to use very limited knowledge on the agents to achieve the said improvement. Other possible directions include data-driven solutions to more intricate problems using the network optimization framework, e.g. fault detection and isolation.

-A Proving Proposition 1

In order to prove the proposition, we use the notion of cursive relations established in [26]:

Definition 4** (Cursive Relations, [26]).**

A set $A\subset\mathbb{R}^{2}$ is called cursive if there exists a curve $\alpha:\mathbb{R}\to\mathbb{R}^{2}$ such that the following conditions hold:

• i)

The set $A$ is the image of $\alpha$.

• ii)

The map $\alpha$ is continuous.

• iii)

The map $\alpha$ satisfies $\lim\limits_{|t|\to\infty}\|\alpha(t)\|=\infty$.

• iv)

$\{t\in\mathbb{R}:\ \exists s\neq t,\ \alpha(s)=\alpha(t)\}$* has measure zero.*

A relation $\varUpsilon$ is called cursive if the set $\{(p,q):\ q\in\varUpsilon(p)\}$ is cursive.

The notion of cursive relations is useful as it can help prove that systems are MEIP. Specifically,

Theorem 4** ( [26]).**

A monotone cursive relation is maximally monotone.

We can now prove Proposition 1:

Proof.

Consider an arbitrary steady-state of the system. As $h$ is continuous and strictly monotone ascending, hence invertible, we must have $\dot{x}=0$ for any steady-state input-output pair. Thus, we conclude that any steady-state input-output pair can be written as $(f(\sigma)/g(\sigma),h(\sigma))$ for some $\sigma\in\mathbb{R}$. We first show passivity with respect to every steady-state, and then show that the steady-state input-output relation is maximally monotone. Take a steady-state $(f(x_{0})/g(x_{0}),h(x_{0}))$ of the system, and define $S(x)=\int_{x_{0}}^{x}\frac{h(\sigma)-h(x_{0})}{g(\sigma)}d\sigma$. We claim that $S$ is a storage function for the steady-state input-output pair $(f(x_{0})/g(x_{0}),h(x_{0}))$. Indeed, $S(x)\geq 0$, with equality only at $x_{0}$, immediately follows from strict monotonicity of $h$ and $g>0$. As for the inequality defining passivity, we have:

[TABLE]

where the second term is negative as $\frac{f}{g},h$ are monotone ascending, and the first term is $(y-h(x_{0}))(u-\frac{f(x_{0})}{g(x_{0})})$. Hence, the system is indeed passive with respect to any steady-state input-output pair. As for maximal monotonicity of the steady-state relation, we recall that it can be parameterized as $(f(\sigma)/g(\sigma),h(\sigma))$ for $\sigma\in\mathbb{R}$. We claim that this relation is both monotone and cursive, which will prove maximal monotonicity. Monotonicity holds as for any $x_{0},x_{1}$,

[TABLE]

due to strict monotonicity. As for cursiveness, the map $\sigma\mapsto(f(\sigma)/g(\sigma),h(\sigma))$ is a curve whose image is the relation. Moreover, it is clear that the map is continuous, and also injective due to (8). Lastly, we have

[TABLE]

so the proof is complete by Theorem 4. ∎

-B Proving Theorem 3.

We first state and prove the following lemma:

Lemma 4**.**

Suppose that the assumptions of Theorem 3 hold, and let $C>0$ be any constant. Define $A_{1}=\{\mathrm{y}\in\mathbb{R}^{n}:\ \|\mathcal{E}^{T}\mathrm{y}-\zeta^{\star}\|\leq C\}$ and $A_{2}=\{\mathrm{y}\in\mathbb{\mathbb{R}}^{n}:\ \sum_{i}k_{i}^{-1}(\mathrm{y}_{i})=0\}$. Then the set $A_{1}\cap A_{2}$ is bounded.

Proof.

First, we note that the inequality $\|\mathcal{E}^{T}\mathrm{y}-\zeta^{\star}\|\leq C$ implies that for any edge $\{i,j\}\in\mathbb{E}$, we have $|\mathrm{y}_{i}-\mathrm{y}_{j}|\leq C+||\zeta^{\star}||$ by the triangle inequality. We let $\omega=(C+||\zeta^{\star}||)\mathrm{diam}(\mathcal{G})$, where $\mathrm{diam}(\mathcal{G})$ is the diameter of the graph $\mathcal{G}$, so that if there exists some $i,j\in\mathbb{V}$ such that $|\mathrm{y}_{i}-\mathrm{y}_{j}|>\omega$ then $\mathrm{y}\not\in A_{1}$. Moreover, let $\mathrm{z}=k(0)$, so $\sum_{i}k_{i}^{-1}(\mathrm{z}_{i})=0$, so that if $\mathrm{y}\in\mathbb{R}^{n}$ satisfies $\forall i:\ \mathrm{y}_{i}>\mathrm{z}_{i}$, then $\mathrm{y}\not\in A_{2}$. Indeed, for each $i$ we have $k_{i}^{-1}(\mathrm{y}_{i})\geq k^{-1}_{i}(\mathrm{z}_{i})$, and $k_{i_{0}}^{-1}(\mathrm{z}_{i_{0}})>k_{i_{0}}^{-1}(\mathrm{y}_{i_{0}})$, meaning that $\sum_{i}k_{i}^{-1}(\mathrm{y}_{i})>\sum_{i}k_{i}^{-1}(\mathrm{z}_{i})=0$. Similarly, if $\forall i,\mathrm{z}_{i}>\mathrm{y}_{i}$ then $\mathrm{y}\not\in A_{2}$. We claim that for any $\mathrm{y}\in A_{1}\cap A_{2}$ and any $i\in\mathbb{V}$, we have $C_{1}<\mathrm{y}_{i}<C_{2}$, where $C_{1}=\min_{j}\mathrm{z}_{j}-\omega-1$ and $C_{2}=\max_{j}\mathrm{z}_{j}+\omega+1$. Indeed, take any $\mathrm{y}\in\mathbb{R}^{n}$, and suppose that $\mathrm{y}_{i}\geq C_{2}$ for some $i\in\mathbb{V}$. There are two possibilities.

• •

There is some $k\in\mathbb{V}$ such that $\mathrm{y}_{k}<\max_{j}\mathrm{z}_{j}+1$. Then $|\mathrm{y}_{i}-\mathrm{y}_{k}|>\omega$, implying that $\mathrm{y}\not\in A_{1}$.

• •

For any $k\in\mathbb{V}$, $\mathrm{y}_{k}\geq\max_{j}\mathrm{z}_{j}+1$, implying that $\mathrm{y}\not\in A_{2}$.

Similarly, one shows that if there is some $i$ such that $\mathrm{y}_{i}\leq C_{1}$, then $\mathrm{y}\not\in A_{1}\cap A_{2}$. This completes the proof. ∎

Proof of Theorem 3.

Consider the solution $\mathrm{y}(a)$ of $0=k^{-1}(\mathrm{y})+\mathcal{E}\mathrm{diag}(a)\gamma(\mathcal{E}^{T}\mathrm{y})$ as a function of $a$. Then $\mathrm{y}(a)$ is a differentiable function by the inverse function theorem, and its differential is given by $\frac{d\mathrm{y}}{da}=-X(\mathrm{y}(a))\mathcal{E}\mathrm{diag}(\gamma(\mathcal{E}^{T}\mathrm{y}(a))),$ where the matrix $X(\mathrm{y})$ is given by

[TABLE]

We note that $X(\mathrm{y})$ is positive-definite for any $\mathrm{y}\in\mathbb{R}^{n}$, by Proposition 2 in [36]. Thus, the gradient of $F$ is given by:

[TABLE]

We note that $v^{T}\mathrm{diag}(\gamma(\mathcal{E}^{T}\mathrm{y}(a))=\mathcal{E}^{T}\mathrm{y}(a)-\zeta^{\star}$, as $\gamma_{e}(x_{e})=0$ if and only if $x_{e}=\zeta^{\star}_{e}$ by strict monotonicity. Thus,

[TABLE]

which is non-positive as $X(\mathrm{y}(a))$ is a positive-definite matrix.

Now, we claim that $v^{T}\nabla F(a)=0$ if and only if $\mathcal{E}^{T}\mathrm{y}(a)=\zeta^{\star}$. Indeed, $\zeta^{\star}\in\mathrm{Im}(\mathcal{E}^{T})$, so we denote $\zeta^{\star}=\mathcal{E}^{T}\mathrm{y}_{0}$ for some $\mathrm{y}_{0}$. As $X(\mathrm{y}(a))$ is positive definite, (12) implies that $v^{T}\nabla F(a)=0$ if and only if $\mathcal{E}(\mathcal{E}^{T}\mathrm{y}(a)-\zeta^{\star})=\mathcal{E}\mathcal{E}^{T}(\mathrm{y}(a)-\mathrm{y}_{0})$ is the zero vector. The kernel of the Laplacian $\mathcal{E}\mathcal{E}^{T}$ is the span of the all-one vector $\mathbb{1}_{n}$, so $\mathrm{y}(a)-\mathrm{y}_{0}=\kappa\mathbb{1}_{n}$ for some $\kappa$, hence $\mathcal{E}^{T}\mathrm{y}(a)=\mathcal{E}^{T}\mathrm{y}_{0}=\zeta^{\star}$. This concludes the first part of the proof.

As for convergence, we know that if $h$ is small enough, then $F(a^{(j+1)})<F(a^{(j)})$. However, the value of $h$ so that $F(a^{(j+1)})<F(a^{(j)})$ can depend on $a^{(j)}$ itself, but it is obvious that if $h$ is small enough, then for any $j$, we have $F(a^{(j)})\leq F(a^{(0)})$. We let $C=F(a^{(0)})=\|\mathcal{E}^{T}\mathrm{y}(a^{(0)})-\zeta^{\star}\|$, and consider the sets $A_{1}=\{\mathrm{y}:\ \|\mathcal{E}^{T}\mathrm{y}-\zeta^{\star}\|\leq C\}$ and $A_{2}=\{\mathrm{y}:\ \sum_{i}k_{i}^{-1}(\mathrm{y}_{i})=0\}$. For any $j$, we know that $\mathrm{y}(a^{(j)})\in A_{1}$ by above, and that $\mathrm{y}(a^{(j)})\in A_{2}$ by the steady-state equation $0=k^{-1}(\mathrm{y}(a))+\mathcal{E}\mathrm{diag}(a)\gamma(\mathcal{E}^{T}\mathrm{y}(a))$. Thus, all steady-state outputs achieved during the algorithm are in the set $\mathcal{D}=A_{1}\cap A_{2}$, which is bounded by Lemma 4. The map sending a matrix to its minimal singular value is continuous, meaning that $\underline{\sigma}(X(\mathrm{y}))$ achieves a minimum on the set $\mathcal{D}$ at some point $\mathrm{y}_{1}$, and the minimum is positive as $X(\mathrm{y}_{1})$ is positive-definite. We denote the minimum value by $\underline{\sigma}(\mathcal{D})$.

Now, consider equation (12). We get that $v^{T}\nabla F(a)$ is bounded by above $-\underline{\sigma}(\mathcal{D})||\mathcal{E}(\mathcal{E}^{T}\mathrm{y}(a)-\zeta^{\star})||^{2}$. In turn, we saw above that unless $\mathcal{E}^{T}\mathrm{y}(a)=\zeta^{\star}$, $\mathcal{E}(\mathcal{E}^{T}\mathrm{y}(a)-\zeta^{\star})\neq 0$, meaning that $\|\mathcal{E}(\mathcal{E}^{T}\mathrm{y}(a)-\zeta^{\star})\|\geq\varsigma||\mathcal{E}^{T}\mathrm{y}(a)-\zeta^{\star}||^{2}$, where $\varsigma$ is the minimal nonzero singular value of $\mathcal{E}$. Hence, at any time step $j$, $v^{T}\nabla F(a^{(j)})<-\underline{\sigma}(\mathcal{D})\varsigma F(a^{(j)})$. In turn we conclude that $F(a^{(j+1)})=F(a^{(j)})-h\underline{\sigma}(\mathcal{D})\varsigma F(a^{(j)})+\mathcal{O}(h)=(1-h\underline{\sigma}(\mathcal{D})\varsigma)F(a^{(j)})+\mathcal{O}(h)$. Iterating this equation shows that eventually, $F(a^{(j)})<\varepsilon$, completing the proof. ∎