A system-theoretic framework for privacy preservation in continuous-time multiagent dynamics

TL;DR

This paper introduces a system-theoretic framework called dynamical privacy for protecting initial states in multiagent systems by using local, time-varying output masks that asymptotically reveal true states.

Contribution

It proposes a novel privacy-preserving method using output masks that are local and time-varying, ensuring privacy while maintaining system stability and convergence.

Findings

Masked system converges to the same equilibrium as the original system.

Existence of equilibrium points in masked systems conflicts with privacy preservation.

Application demonstrated on social opinion, consensus, and synchronization models.

Abstract

In multiagent dynamical systems, privacy protection corresponds to avoid disclosing the initial states of the agents while accomplishing a distributed task. The system-theoretic framework described in this paper for this scope, denoted dynamical privacy, relies on introducing output maps which act as masks, rendering the internal states of an agent indiscernible by the other agents as well as by external agents monitoring all communications. Our output masks are local (i.e., decided independently by each agent), time-varying functions asymptotically converging to the true states. The resulting masked system is also time-varying, and has the original unmasked system as its limit system. When the unmasked system has a globally exponentially stable equilibrium point, it is shown in the paper that the masked system has the same point as a global attractor. It is also shown that existence of equilibrium points in the masked system is not compatible with dynamical privacy. Application of dynamical privacy to popular examples of multiagent dynamics, such as models of social opinions, average consensus and synchronization, is investigated in detail.

Equations

$$\dot{x}=g(t,x),x(t_o)=x_o$$

$$\dot{x}=f(x),x(0)=x_o,$$

$${\dot{x}}_i=f_i(x_i,x_j,j\in {\mathcal{N}}_i),i=1,\dots ,n$$

$$\begin{array}{rl}{f}_i(x_i,x_j,j\in {\tilde{\mathcal{N}}}_i)\ne f_i(x_i,x_j,j\in {\mathcal{N}}_i)& \forall {\tilde{\mathcal{N}}}_i⊊{\mathcal{N}}_i,\\ & \forall i=1\dots ,n.\end{array}$$

$$\begin{array}{rl}h:{\mathbb{R}}_{+}\times {\mathbb{R}}^n\times {\mathbb{R}}^m& \to {\mathbb{R}}^n\\ (t,x,\pi )& \mapsto y(t)=h(t,x(t),\pi )\end{array}$$

$$\dot{x}$$

$$y$$

$$h_i(t,x_i,{\pi }_i)=(1+{\varphi }_i{e}^{-{\sigma }_{i}t})x_i,{\varphi }_i\ge 0,{\sigma }_i>0$$

$$h_i(t,x_i,{\pi }_i)=x_i+{\gamma }_i{e}^{-{\delta }_{i}t},{\delta }_i>0,{\gamma }_i\ne 0$$

$$h_i(t,x_i,{\pi }_i)=c_i(x_i+{\gamma }_i{e}^{-{\delta }_{i}t}),c_i>1,{\delta }_i>0,{\gamma }_i\ne 0$$

$$\begin{array}{rl}{h}_i(t,x_i,{\pi }_i)=& (1+{\varphi }_i{e}^{-{\sigma }_{i}t})(x_i+{\gamma }_i{e}^{-{\delta }_{i}t}),\\ & {\varphi }_i>0,{\sigma }_i>0,{\delta }_i>0,{\gamma }_i\ne 0\end{array}$$

$$h(t,x,\pi )=(I+\Phi e^{-\Sigma t})(x+e^{-\Delta t}\gamma )$$

$${\dot{x}}_i$$

$$y_i$$

$$h(t,x,p)=C(x+e^{-\Delta t}\gamma ),$$

$$\dot{x}=f(C^{-1}x).$$

$$\dot{x}=-x+\kappa A\psi (x)$$

$$\dot{V}=-x^{T}x+\kappa x^{T}A\psi (x)\le x^T(-I+\kappa A)x<0$$

$$\dot{x}=-(L+\Theta )x+\Theta x_o,x(0)=x_o,$$

$$\dot{z}=-(L+\Theta )z,$$

$$\begin{array}{rl}\dot{x}& =(-L-\Theta )y+\Theta y_o\\ y& =h(t,x,\pi )=\left(I+\Phi e^{-\Sigma t}\right)\left(x+e^{-\Delta t}\gamma \right)\end{array}$$

$$\dot{x}=-Lx,x(0)=x_o$$

$$\begin{array}{rl}\dot{x}& =-Ly\\ y& =h(t,x,\pi )=\left(I+\Phi e^{-\Sigma t}\right)\left(x+e^{-\Delta t}\gamma \right).\end{array}$$

$$h_i(t,x_i,{\pi }_i)=(I+{\Phi }_i{e}^{-{\Sigma }_{i}t})(x_i+e^{-{\Delta }_{i}t}{\gamma }_i)$$

$${\dot{x}}_i$$

$$\dot{s}=f(s).$$

$$\underset{t\to \infty }{\lim }\Vert x_i(t)-s(t)\Vert =0\forall x_i(0)\in {\mathbb{R}}^{\nu },\forall i=1,\dots ,n.$$

$$\|f(x)-f(z)\|\leq q\big{(}(x-z)^{T}R(x-z)\big{)}^{\frac{1}{2}}\qquad\forall\;x,\,z\in\mathbb{R}^{\nu}$$

$$q\Xi \otimes R-\left(\frac{1}{2}(\Xi L+L^T\Xi )+\Xi P\right)\otimes R<0$$

$${\dot{x}}_i$$

Full text

††thanks: Work supported in part by a grant from the Swedish Research Council (grant n. 2015-04390). A preliminary version of this paper was presented at CDC’19 [3].

1 Introduction

Most multiagent systems rely intrinsically on collaboration among agents in order to accomplish a joint task. Collaboration however means that exchange of information among the agents cannot be dispensed with. If the information is sensitive, then questions like respecting the privacy of the individual agents naturally rise. Several approaches exist to address this conundrum of exchanging information without revealing it. One approach is called differential privacy [13, 14] and consists, roughly speaking, in corrupting the information being transmitted with a noise from an appropriate distribution so that an observer accessing the transmitted signals can only reconstruct the original data up to a prespecified precision level. Another approach relies on cryptography. Encrypted messages can be exchanged among the agents in various ways, e.g. through trusted third parties [25], obfuscation [4], or through distributed cryptography schemes [43]. In these approaches the messages from each agent (corrupted with noise or encrypted) are typically exchanged through a communication graph and hence they are available to the other agents of the network. Only the protection mechanism (noise source or cryptographic scheme) is kept private by each agent.

Both approaches have been recently used for multiagent dynamical systems [11, 19, 21, 34, 35, 43, 50]. In this case the information to keep private is typically the initial state of the agents. A problem that is often studied in this context is the consensus problem, because it can be used as a basic building block in many distributed algorithms in database computations, sensor fusion, load balancing, clock synchronization, etc. Dynamically, a consensus scheme consists of a stable system in which the final value reached asymptotically is the (weighted) mean of the initial conditions of the agents. A privacy protected consensus should render this value available to all agents while not disclosing the initial conditions themselves to the other agents. For instance, differentially private average consensus schemes are proposed in [18, 21, 34]. Clearly the addition of noise impacts also the performances of the consensus algorithm: convergence to the true value might be missing [21] or be guaranteed only in expectation [34]. Many other variants are possible: for instance in [20, 29, 31, 41], a non-stochastic perturbation is injected at the nodes, with the constraint that the sum (or integral) over time vanishes. A cryptography-based approach requires instead one or more layers of data encryption technology which must themselves be kept secure and protected [25, 43]. Other system-oriented approaches to privacy protection in distributed computations appear e.g. in [2, 12, 16, 23, 27, 32, 39, 52].

The aim of this paper is to propose a conceptually different framework for privacy preservation of the initial states of multiagent dynamics, inspired by system-theoretic considerations. Our framework is exact, and is developed for continuous-time dynamical systems. It relies on what we call output masks, i.e., local (in the sense of “agent-local”, that is, decided and implemented independently by each agent) time-varying transformations of the states to be transmitted to the neighboring nodes, whose functional form and/or numerical parameters are kept hidden to the other agents. In the privacy literature, the use of masking maps is widespread. For instance, non-invertible maps are used in homomorphic or semi-homomorphic encryption [24, 15, 25, 43], as well as in secure wiretap channels [51]. In the present context, output masks are used to offset the initial condition in a way such that an eavesdropping (curious but not malicious) agent cannot reconstruct it, neither directly nor using a model of the system. In fact, even when an eavesdropper has knowledge of the vector field used by an agent, reconstruction of the initial state of that agent requires to set up a state observer, which in turn requires to identify the functional form and the numerical parameters of the output mask of the agent. In the paper this joint “system identification” and “initial state detection” problem is called discernibility, and conditions are given that render the initial state indiscernible.

The approach we follow (offsetting the initial condition) is somewhat related to [20, 31, 41]. However, our use of output masks enables us to carry out a thorough analysis of the dynamical properties of the masked system, which is novel and insightful of the implications of preserving privacy on the dynamics. When the original unmasked system is globally exponentially stable, or perhaps exponentially stable on “slices” of the state space if there is a continuum of equilibria, as in the consensus problem, we show in the paper that under the assumption that no agent has in-neighborhood that covers that of another agent [41, 31], the masked multiagent system globally uniformly converges to the same attractor as the unmasked system while guaranteeing the privacy of the initial conditions at any level of precision.

The price to pay for guaranteeing privacy is that the masked system is time-varying and has no fixed points. However, as long as the output masks are constructed to converge asymptotically to the unmasked state, the masked time-varying system has the original system as its limit system [6, 7]. When the unmasked system is autonomous, the resulting masked time-varying system is a case of a so-called asymptotically autonomous system [6, 30].

In spite of the indiscernibility of the initial conditions, the asymptotic collapse of the masked dynamics to the original dynamics guarantees that the distributed computation is carried out correctly anyway. Clearly, dealing with a distributed computation representable as a converging dynamical system is a key prerequisite of our method, hence we refer to it as dynamical privacy.

The system-theoretical framework for dynamical privacy developed in this paper is for continuous-time multiagent dynamics. Unlike [41], where a similar setting is chosen, we do not require the time integrals of the perturbations to be vanishing asymptotically, which gives us more freedom in the choice of the masks and leads to a framework applicable to a broad range of distributed multiagent scenarios.

In the paper we investigate the effect of output masks on three different case studies: a globally exponentially stable nonlinear system, an average consensus problem, and a system of diffusively coupled higher order ODEs achieving pinning synchronization [10, 53, 54]. In all three cases a privacy preserving version of the system based on output masks is shown to have the equilibrium point of the unmasked system as unique attractor. However, as the masked system lacks fixed points, it cannot be stable at the attractor. This behavior is designed in purpose. Think for instance at a situation in which the initial conditions are all in a neighborhood of the (say, globally stable) equilibrium point of the unmasked system. If the masked system is stable around that point, its trajectories remain confined in a neighborhood of the equilibrium for all times, leading to an approximate disclosure of the initial states. In order to avoid such situations, a masked system cannot preserve neighborhoods of its attractor, or, in other words, the attractor cannot be also an equilibrium point. To achieve this, our output masks have to be inhomogeneous in the state variables. Such structure is reminiscent of the additive noise used e.g. in differential privacy.

Technically, to show global attractivity in the masked system (in the complement of the agreement subspace for consensus-like problems), we use Lyapunov arguments. The Lyapunov function of the unmasked system is shown to lead to Lyapunov derivatives which are in general sign indefinite, but upper bounded by terms that decay to [math] as $t\to\infty$ [33]. The reasoning is fundamentally different from those used in stability analysis of time-varying systems [1, 26, 28], but somehow related to constructions used in input-to-state stability [47] and in the stability analysis of nonlinear systems in presence of additive exponentially decaying disturbances [48]. In particular, our masked system has a so-called converging-input converging-state property [45]. Boundedness of its trajectories is imposed by choosing Lyapunov functions with globally bounded gradients [46, 49]. The argument is reminiscent of those used in cascade systems [9, 37, 44] or in observer-based nonlinear control [5].

While the importance of initial conditions is well-known in problems such as average consensus (the final value changes with the initial condition, hence privacy questions are self-evident) in the paper we show that similar privacy issues may arise also in other cases in which the unmasked system is globally exponentially stable. In particular we show that in continuous-time Friedkin-Johnsen models of opinion dynamics [40], the value of the equilibrium point is also a function of the initial conditions, because an inhomogeneous term, depending on the initial conditions, is added to an asymptotically stable linear system. Clearly this is a context in which non-disclosure of the initial states could be of strong relevance.

The case of pinned synchronization is instead an example of an unmasked system which is time-varying (it depends on the pinning exosystem [53, 54]). Our privacy protection framework applies also to this case, the only difference being that the limit system of the masked system is itself time-varying.

The rest of the paper is organized as follows: a few preliminary results are outlined in Section 2, while the dynamical privacy problem and the properties of the output masks are formulated in Section 3. In Section 4 the case of a globally exponentially stable unmasked system (and the related case of Friedkin-Johnsen opinion dynamics model) is discussed. Sections 5 and 6 deal with privacy preservation respectively for the average consensus problem and for a pinning synchronization problem. The proofs of all results are gather in the Appendix.

In the conference version of this paper, [3], only the average consensus problem of Section 5 is discussed. The material of Sections 4 and 6 is presented here for the first time.

2 Preliminaries

A continuous function $\alpha\,:\,[0,\infty)\to[0,\,\infty)$ is said to belong to class $\mathcal{K}_{\infty}$ if it is strictly increasing and $\alpha(0)=0$. Subclasses of $\mathcal{K}_{\infty}$ which are homogeneous polynomials of order $i$ will be denoted $\mathcal{K}_{\infty}^{i}$: $\alpha(r)=ar^{i}$ for some constant $a>0$. A continuous function $\zeta\,:\,[0,\infty)\to[0,\,\infty)$ is said to belong to class $\mathcal{L}$ if it is decreasing and $\lim_{t\to\infty}\zeta(t)=0$. In particular, we are interested in $\mathcal{L}$ functions that are exponentially decreasing: $\zeta(t)=ae^{-\delta t}$ for some $a>0$ and $\delta>0$. We shall denote such subclass $\mathcal{L}^{e}\subset\mathcal{L}$. A continuous function $\beta\,:\,[0,\infty)\times[0,\,\infty)\to[0,\,\infty)$ is said to belong to class $\mathcal{KL}_{\infty}^{i,e}$ if the mapping $\beta(r,\,t)$ belongs to class $\mathcal{K}_{\infty}^{i}$ for each fixed $t$ and to class $\mathcal{L}^{e}$ for each fixed $r$, i.e., $\beta(r,\,t)=ar^{i}e^{-\delta t}$ for some $a>0$ and $\delta>0$.

Consider

[TABLE]

where $g\,:\,\mathbb{R}_{+}\times\mathbb{R}^{n}\to\mathbb{R}^{n}$ is Lipschitz continuous in $x$, measurable in $t$, and such that for each $x_{o}\in\mathbb{R}^{n}$ and each $t_{o}\in\mathbb{R}_{+}$ the solution of (1), $x(t,\,x_{o})$, exists in $[0,\,\infty)$. A point $x^{\ast}\in\mathbb{R}^{n}$ is an equilibrium point of (1) if $g(t,\,x^{\ast})=0$ for a.e.111almost every, i.e., except for at most a set of Lebesgue measure [math]. $t\geq t_{o}$.

A point $x^{\ast}\in\mathbb{R}^{n}$ is uniformly globally attractive for (1) if for each $\nu>0$ there exists $T=T(\nu)>0$ such that for each solution $x(t,x_{o})$ of (1) it holds that $\|x(t,\,x_{o})-x^{\ast}\|<\nu$ for each $t>t_{o}+T$, each $x_{o}\in\mathbb{R}^{n}$ and each $t_{o}\geq 0$. In particular, if $x^{\ast}$ is a uniform global attractor for (1), then as $t\to\infty$ all trajectories $x(t,x_{o})$ converge to $x^{\ast}$ uniformly in $t$ for all $t_{o}\geq 0$ and $x_{o}$. A point $x^{\ast}$ can be attractive for (1) without being an equilibrium of (1) (we will use this fact extensively in the paper).

Given (1), denote $g_{s}(t,\,x)$ the translate of $g(t,\,x)$: $g_{s}(t,\,x)=g(t+s,\,x)$. A (possibly time-dependent) system $\dot{x}=\tilde{g}(t,\,x)$ is called a limit system of (1) if there exists a sequence $\{s_{k}\}$, $s_{k}\to\infty$ as $k\to\infty$, such that $g_{s_{k}}(t,\,x)$ converges to $\tilde{g}(t,\,x)$ [6]. An existence condition for a limit system $\tilde{g}(t,\,x)$ is given in Lemma 1 of [26]: when $g(t,\,x)$ is a uniformly continuous and bounded function, then there exist increasing and diverging sequences $\{s_{k}\}$ such that on compact subsets of $\mathbb{R}^{n}$ $g_{s_{k}}(t,\,x)$ converges uniformly to a continuous limit function $\tilde{g}(t,\,x)$ on every compact of $[0,\,\infty)$, as $k\to\infty$. In general the limit system may not be unique nor time-invariant. However, when it exists unique, then it must be autonomous [6, 42] because all translates $g_{s+s^{\prime}}(t,\,x)$ must have themselves a limit system hence the latter cannot depend on time. The time-varying system (1) is called asymptotically autonomous in this case.

The $\omega$-limit set of $x(t,\,x_{o})$, denoted $\Omega_{x_{o}}$, consists of all points $x^{\ast}$ such that a sequence $\{t_{k}\}$, with $t_{k}\to\infty$ when $k\to\infty$, exists for which $\lim_{k\to\infty}x(t_{k},\,x_{o})=x^{\ast}$. For time-varying systems, if a solution is bounded then the corresponding $\Omega_{x_{o}}$ is nonempty, compact and approached by $x(t,x_{o})$. However, it need not be invariant. Only for limit systems the invariance property may hold, although not necessarily (it may fail even for asymptotically autonomous systems, see [6]).

3 Problem formulation

Consider a distributed dynamical system on a graph with $n$ nodes:

[TABLE]

where $x=\begin{bmatrix}x_{1}&\ldots&x_{n}\end{bmatrix}^{T}\in\mathbb{R}^{n}$ is a state vector and $f=[f_{1}\,\ldots f_{n}]^{T}:\,\mathbb{R}^{n}\to\mathbb{R}^{n}$ is a Lipschitz continuous vector field. Standing assumptions in this paper are that (2) possesses a unique solution continuable on $[0,\,\infty)$ for all $x_{o}\in\mathbb{R}^{n}$ and that information can be exchanged only between first neighbors on the graph, i.e.,

[TABLE]

with $\mathcal{N}_{i}$ the in-neighborhood of node $i$. Furthermore, to avoid trivial situations, we impose that $\mathcal{N}_{i}$ is the “essential neighborhood” of agent $i$ [31], i.e.,

[TABLE]

We are interested in cases in which the system (3) has a globally exponentially stable equilibrium point, i.e., $\lim_{t\to\infty}x(t)=x^{\ast}$ for all $x_{o}$, but also in cases in which the presence of a conservation law (as in the consensus problem) leads to exponential stability on some submanifold depending on the initial conditions, i.e., $\lim_{t\to\infty}x(t)=x^{\ast}(x_{o})$.

The privacy preservation problem consists in using a system like (2) to perform the computation of $x^{\ast}$ in a distributed manner, while avoiding to divulgate the initial condition $x_{o}$ to the other nodes. Clearly this cannot be achieved directly on the system (2) which is based on exchanging the values $x_{i}$ between the nodes. It can however be achieved if we insert a mask on the value $x(t)$ which preserves convergence to $x^{\ast}$, at least asymptotically. The masks we propose in this paper have the form of time-varying output maps.

3.1 Output masks

Consider a continuously differentiable time-varying output map

[TABLE]

where $y=\begin{bmatrix}y_{1}&\ldots&y_{n}\end{bmatrix}^{T}\in\mathbb{R}^{n}$ is an output vector of the same size as $x$, and $\pi\in\mathbb{R}^{m}$ is a vector of parameters splittable into $n$ subvectors (not necessarily of the same dimension), one for each node of the network: $\pi=\{\pi_{1},\ldots,\pi_{n}\}$.

In the following we refer to $h(t,x(t),\pi)$ as an output mask and to $y$ as a masked output. The state $x$ of the system is first masked into $y$ and then sent to the first out-neighbors on the graph. The original system (2) can therefore be modified into the following masked system:

[TABLE]

Denote $y(t,x_{o})$, of components $y_{i}(t,x_{o,i})$, $i=1,\ldots,n$, the output trajectory of (6) from the initial state $x_{o}$, of components $x_{o,i}$. We assume in what follows that the vector field $f(\cdot)$ is publicly known (i.e., each agent knows the shape of the functions $f_{1}(\cdot),\ldots,f_{n}(\cdot)$) and that each node knows the output trajectories $y_{i}(t,x_{o,i})$ of its in-neighbors. The state $x$ and the output mask $h(t,x,\pi)$ (functional form plus values of the parameters $\pi$) are instead private to each agent, as explained more in detail next.

Definition 1

A $C^{1}$ output map $h$ is said a local mask if it has components that are local, i.e.,

1. P1:

$h_{i}(t,x,\pi)=h_{i}(t,x_{i},\pi_{i})\qquad i=1,\ldots,n$.

The property of locality guarantees that the output map $h_{i}$ can be independently decided by node $i$. Both the functional form chosen for $h_{i}(\cdot)$ and the numerical value of the parameters $\pi_{i}$ can therefore remain hidden to the other agents.

The output mask needs also to avoid mapping neighborhoods of a point $x^{\ast}$ of (2) (typically an equilibrium point) into themselves. For that, we introduce the following definition.

Definition 2

A $C^{1}$ output map $h$ is said to preserve neighborhoods of a point $x^{\ast}$ if, for all small $\epsilon>0$, $\|x_{o}-x^{\ast}\|<\epsilon\;\;\Longrightarrow\;\;\|h(0,x_{o},\pi)-x^{\ast}\|<\epsilon$. It is said not to preserve neighborhoods otherwise.

These notions are used in the following definition.

Definition 3

A $C^{1}$ output map $h$ is said a privacy mask if it is a local mask and in addition

1. P2:

$h_{i}(0,x_{i},\pi_{i})\neq x_{i}$* $\forall\;x_{i}\in\mathbb{R}^{n}$, $i=1,\ldots,n$;* 2. P3:

$h(t,x,\pi)$* does not preserve neighborhoods of any $x\in\mathbb{R}^{n}$;* 3. P4:

$h_{i}(t,x_{i},\pi_{i})$* is strictly increasing in $x_{i}$ for each fixed $t$ and $\pi_{i}$, $i=1,\ldots,n$.*

Property P2 means that $h_{i}(\cdot)$ has no fixed points. Property P4 resembles a definition of $\mathcal{K}_{\infty}$ function, but it is in fact more general: $x=0$ is not a fixed point of $h$ for any finite $t$, and $h$ need not be nonnegative in $x$. Monotonicity of $h$ in $x$ (for each fixed $\pi$) follows from Property P4 combined with P1. It implies that $h$ is a bijection in $x$ for each fixed $t$ and $\pi$, although one that does not preserve the origin. This is meant to avoid that the output mask introduces undesired behavior in the system, like spurious equilibrium points.

In many cases, it will be necessary to impose that the privacy mask converges asymptotically to the true state, i.e., that the perturbation induced by the mask is vanishing.

Definition 4

The output map $h$ is said a vanishing privacy mask if it is a privacy mask and in addition

1. P5:

$|h_{i}(t,x_{i},\pi_{i})-x_{i}|$* is decreasing in $t$ for each fixed $x_{i}$ and $\pi_{i}$, and $\lim_{t\to\infty}h_{i}(t,x_{i},\pi_{i})=x_{i}$ for each fixed $\pi_{i}$, $i=1,\ldots,n$.*

The difference between the true initial condition $x_{o,i}$ and the masked output $h_{i}(0,x_{o,i},\pi_{i})$ can be used to quantify the level of privacy for agent $i$. More formally, if $h_{i}(\cdot)$ is a privacy mask for agent $i$, we denote $\rho_{i}(x_{o,i})=|h_{i}(0,x_{o,i},\pi_{i})-x_{o,i}|$ the privacy metric of agent $i$ relative to the initial condition $x_{o,i}$, and $\rho(x_{o})=\min_{i=1,\ldots,n}\rho_{i}(x_{o,i})$ the privacy metric of the system relative to the initial condition $x_{o}$.

3.2 Examples of output masks

The following are examples of output masks.

Linear mask

[TABLE]

(i.e., $\pi_{i}=\{\phi_{i},\,\sigma_{i}\}$). This local vanishing mask is not a proper privacy mask since $h_{i}(0,0,\pi_{i})=0$ i.e. the origin is not masked. Notice that all homogeneous maps have this problem (and they fail to escape neighborhoods of $x_{i}$).

Additive mask

[TABLE]

(i.e., $\pi_{i}=\{\delta_{i},\,\gamma_{i}\}$) is a vanishing privacy mask.

Affine mask

[TABLE]

(i.e., $\pi_{i}=\{c_{i},\,\delta_{i},\,\gamma_{i}\}$) is also a privacy mask. Since $\lim_{t\to\infty}h_{i}(t,\,x_{i},\,\pi_{i})=c_{i}x_{i}$, it is however not vanishing.

Vanishing affine mask

[TABLE]

(i.e., $\pi_{i}=\{\phi_{i},\,\sigma_{i},\,\delta_{i},\,\gamma_{i}\}$). This privacy mask is also vanishing. Notice that in vector form, assuming all nodes adopt it, the vanishing affine mask can be expressed as

[TABLE]

where $\Phi={\rm diag}(\phi_{1},\ldots,\phi_{n})$, $\Sigma={\rm diag}(\sigma_{1},\ldots,\sigma_{n})$, $\Delta={\rm diag}(\delta_{1},\ldots,\delta_{n})$, and $\gamma=\begin{bmatrix}\gamma_{1}&\ldots&\gamma_{n}\end{bmatrix}^{T}$.

The following proposition shows that for these masks the level of privacy (as defined by the metric $\rho$), can be made arbitrary if each agent chooses $\pi_{i}$ in an appropriate way.

Proposition 1

Given $\lambda>0$, for each of the privacy masks (8), (9) and (10) the parameters $\pi_{i}$ can be chosen locally by each agent $i$, $i=1,\ldots,n$, so that $\rho(x_{o})>\lambda$ for any $x_{o}$.

By definition of $\rho$, the choice of parameters $\pi_{i}$ in Proposition 1 is compatible with the privacy of the maps $h_{i}(\cdot)$ (each agents knows its own $x_{o,i}$, hence can compute $\rho_{i}(x_{o,i})$ without disclosing $x_{o,i}$).

3.3 Dynamically private systems

Consider the system (6), rewritten here for convenience in components ($i=1,\ldots,n$):

[TABLE]

We would like to understand when an eavesdropping agent $j$ can violate the privacy of agent $i$, estimating its initial condition $x_{o,i}$. Recall that we are assuming that the agent $j$ knows:

1. K1:

the form of the vector field $f_{i}(\cdot)$, 2. K2:

the output trajectories of its incoming neighborhood: $y_{k}(t,x_{o,k})$, $k\in\{\mathcal{N}_{j}\cup\{j\}\}$, $t\in[t_{o},\,\infty)$,

while instead the following are unknown for agent $j$:

1. U1:

the form of the output mask $h_{i}(\cdot)$ and the numerical values of the parameters $\pi_{i}$, 2. U2:

the output trajectories not in its incoming neighborhood: $y_{k}(t,x_{o,k})$, $k\notin\{\mathcal{N}_{j}\cup\{j\}\}$.

Because of item U1 above, the problem of estimating $x_{o,i}$ from the output in (12) cannot be cast as a state observability problem, but rather it has to be treated as a joint system identification + observability problem. To characterize this unusual situation we introduce a new concept, discernibility.

Definition 5

The initial condition of agent $i$, $x_{o,i}$, is said discernible for agent $j$ ($j\neq i$) if agent $j$ can estimate $x_{o,i}$ from the knowledge of K1 and K2. It is said indiscernible for agent $j$ otherwise. An initial condition $x_{o}$ is said indiscernible if all of its components $x_{o,i}$ are indiscernible for all agents $j\in\{1,\ldots,n\}\setminus\{i\}$.

Indiscernibility refers to the impossibility to solve the joint identification+observation problem of estimating $x_{o,i}$ in (12). It can be imposed using the properties of a privacy mask together with the following Assumption 1 (see [41] and [31], Corollary 1).

Assumption 1

(No completely covering neighborhoods)* The system (6) is such that $\{\mathcal{N}_{i}\cup\{i\}\}\nsubseteq\{\mathcal{N}_{j}\cup\{j\}\}$, $\forall\;i,\,j=1,\ldots,n$, $i\neq j$.*

Assumption 1 guarantees that no node has complete information of what is going on at the other nodes. This is a condition on the topology of the graph, and therefore a system property, rather than simply a property of well-conceived output maps.

Combining indiscernibility with privacy of the output masks, we can formulate the following definition.

Definition 6

The system (6) is called a dynamically private version of (2) if

$h$* is a privacy mask;* 2. 2.

the solution of (6) exists unique in $[0,\,\infty)$ and is bounded $\forall\;x_{o}\in\mathbb{R}^{n}$; 3. 3.

$\lim_{t\to\infty}y(t)=\lim_{t\to\infty}x(t)$; 4. 4.

indiscernibility of the initial condition is guaranteed.

The next proposition relates indiscernibility to Assumption 1.

Proposition 2

If the system (6) satisfies conditions 1-3 of Definition 6 and Assumption 1, then it is a dynamically private version of (2).

Remark 1

From Proposition 1, when a system is dynamically private with any of (8), (9) and (10) as mask, then privacy can be made to hold at an arbitrary level of precision, i.e., given $\lambda>0$, $\rho(x_{o})>\lambda$ can be guaranteed for any $x_{o}$ only though local choices of the parameters $\pi_{i}$ for each agent.

The privacy property P3 of $h(\cdot)$ suggests that in a dynamically private system we cannot have equilibrium points and therefore we cannot talk about stability (of equilibria), while convergence of $y(t)$ to $x(t)$ suggests that as long as $f(\cdot)$ is autonomous, a dynamically private system is asymptotically autonomous with the unmasked system as limit system. This can be shown to be always true if the output mask is vanishing.

Proposition 3

If (6) is a dynamically private version of (2), then it cannot have equilibrium points. Furthermore, if $h(\cdot)$ is a vanishing privacy mask, then the system (6) is asymptotically autonomous with limit system (2).

The “vanishing” attribute of the second part of Proposition 3 is sufficient but not necessary. As we will see below, when (2) is globally exponentially stable, the condition that the output mask must be vanishing can be dispensed with.

4 Dynamical privacy in globally exponentially stable systems

In this section we restrict ourselves to unmasked systems (2) having a globally exponentially stable equilibrium point. Under Assumption 1, any privacy mask (not necessarily vanishing) can guarantee privacy of the initial conditions. We will only show the simplest case of affine mask. Since we rely on standard converse Lyapunov theorems, we also request (2) to be globally Lipschitz.

Theorem 1

Consider the system (2) with $f\,:\,\mathbb{R}^{n}\to\mathbb{R}^{n}$ globally Lipschitz continuous, $f(0)=0$, and the masked system (6) with the affine mask

[TABLE]

$C={\rm diag}(c_{1},\ldots,c_{n})$, $c_{i}>1$, $\Delta={\rm diag}(\delta_{1},\ldots,\delta_{n})$, $\delta_{i}>0$, and $\gamma=\begin{bmatrix}\gamma_{1}&\ldots&\gamma_{n}\end{bmatrix}^{T}$, $\gamma_{i}\neq 0$. If Assumption 1 holds and the equilibrium $x^{\ast}=0$ is globally exponentially stable for (2), then $x^{\ast}=0$ is uniformly globally attractive for the masked system (6). Furthermore, (6) is a dynamically private version of (2).

Remark 2

Even if (2) has $x^{\ast}=0$ as equilibrium point, the masked system (6) does not, as can be seen from the expression (39) in the proof of Theorem 1. This follows from the inhomogeneity of the output mask. Since $x^{\ast}=0$ is not stationary, we cannot talk about stability of its neighborhoods. Nevertheless, $x^{\ast}$ remains an attractor for all trajectories of the system.

The following corollary states that the dynamically private system is asymptotically autonomous with $\omega$-limit set identical to that of the corresponding unmasked system.

Corollary 1

Under the assumptions of Theorem 1, the system (6) with the output mask (13) is asymptotically autonomous with limit system

[TABLE]

The $\omega$-limit set of each trajectory of (6) is given by $\{0\}$ for each $x_{o}\in\mathbb{R}^{n}$.

Notice that since the affine mask (13) is not vanishing, (14) differs from (2) (yet $x^{\ast}$ is the same).

Remark 3

The result of Theorem 1 can be rephrased as a converging-input converging-state property [45]: under the assumption of $f$ (locally) Lipschitz continuous and $x^{\ast}$ globally asymptotically stable, boundedness of the trajectories is enough to guarantee that $x(t)\to 0$ as $t\to\infty$. However, guaranteeing boundedness is a nontrivial task: a globally asymptotically stable system can be destabilized by an additive perturbation which is arbitrarily small in $\mathcal{L}_{1}$ norm [46]. Similarly, a globally exponentially stable system with linear sector growth (as opposed to global Lipschitzianity) can be destabilized by arbitrarily small additive exponentially decaying disturbances [49, 8]. The assumptions made in Theorem 1 imply the boundedness of the gradient of the Lyapunov function, which in turn guarantees boundedness of the solutions.

Example 1

Consider the following interconnected system with saturated nonlinearities

[TABLE]

where the off-diagonal matrix $A\geq 0\;$ is a weighted adjacency matrix of spectral radius $\rho(A)>0$ describing the interactions among the agents and satisfying Assumption 1, $\kappa>0$ is a scalar coefficient, and $\psi(x)=\begin{bmatrix}\psi_{1}(x_{1})&\ldots&\psi_{n}(x_{n})\end{bmatrix}^{T}$, $\psi_{i}(x_{i})=\tanh(x_{i})$, is a vector of saturated sigmoidal functions depending only on the state of the sending node $x_{i}$. The system (15) is used e.g. in [17] to describe collective distributed decision-making systems. If we impose the condition $\kappa<\frac{1}{\rho(A)}$, then $x^{\ast}=0$ is a globally exponentially stable equilibrium point for (15). In fact, in this case a simple quadratic Lyapunov function $V=\frac{1}{2}\|x\|^{2}$ leads to

[TABLE]

because $\psi_{i}(x_{i})$ obeys to the sector inequality $0\leq\psi_{i}(x_{i})x_{i}\leq 1$. Since the system is globally Lipschitz, Theorem 1 is applicable to it if we choose an output mask like (13). Simulations for $n=100$ are shown in Fig. 1 for a privacy measure of $\lambda=1$. Notice in panel (c) how the gap between $x_{o,i}$ and $y_{o,i}$ induced by this privacy level is clearly visible. The initial conditions obey to $\rho_{i}(x_{o,i})=|y_{i}(0)-x_{i}(0)|\geq 1$, but $|y_{i}(t)-x_{i}(t)|$ necessarily decreases as t grows, and converges to 0 as $t$ diverges, see panel (d).

In Example 1 global exponential stability implies that the initial conditions are forgotten asymptotically. In these cases privacy protection might be considered less critical than when the equilibrium point is itself a function of the initial state, as it happens in the next sections.

4.1 Application to continuous-time Friedkin-Johnsen model

Let us consider a continuous-time Friedkin-Johnsen model (also known as Taylor model, see [40])

[TABLE]

where $L$ is an irreducible Laplacian matrix, and $\Theta={\rm diag}(\theta_{1},\ldots,\theta_{n})$, $\theta_{i}\in[0,\,1]$, is a diagonal matrix of so-called susceptibilities, i.e., tendencies of the $i$-th agent to remain attached to its own initial opinion $x_{o_{i}}$. The behavior of the system (16) is analyzed in [40]: when $L$ is irreducible and some $\theta_{i}\neq 0$, it has a single equilibrium point $x^{\ast}=(L+\Theta)^{-1}\Theta x_{o}$ which is asymptotically stable for a solution starting in $x_{o}$. The system reduces to the usual consensus problem when $\theta_{i}=0$ $\forall\,i$ (see Section 5). Notice how in the affine model (16), the initial opinions (initial condition of the system) enter also in the vector field at time $t$. Hence protecting the privacy of the agents in (16) requires a ‘double mask’, i.e., one needs to replace both $x(t)$ and $x_{o}$ with suitably masked versions $y(t)$ and $y_{o}=y(0)$ (since $y_{o}$ is transmitted to the neighboring agents, it can be memorized and used whenever needed).

Denoting $z=x-x^{\ast}=x-(L+\Theta)^{-1}\Theta x_{o}$, then (16) is expressed in $z$ as the linear system

[TABLE]

which has $z^{\ast}=0$ as globally asymptotically (and hence exponentially) stable equilibrium point, meaning that Theorem 1 is applicable. In the original $x$ basis, a consequence of inhomogeneity of (16) is that the attractor $x^{\ast}$ is a function of the initial condition $x_{o}$, and it moves with it: $x^{\ast}=x^{\ast}(x_{o})$. To talk rigorously about global asymptotic stability, we should use (17) in $z$-coordinates. However, for homogeneity of presentation, the next theorem is still formulated in terms of $x$ and $y$ variables, and global asymptotic stability / attractivity is referred to the “moving” point222Unlike for the consensus problem which we will study in Section 5, in this case there is no easy way to describe the orthogonal complement of the space in which $x^{\ast}(x_{o})$ moves. $x^{\ast}(x_{o})$. Another consequence of the inhomogeneous structure of (16) is that novanishing affine privacy masks like the one used in Theorem 1 cannot be used. To obtain convergence to the correct $x^{\ast}(x_{o})$ we need to use a vanishing privacy mask.

Theorem 2

If Assumption 1 holds, the masked system

[TABLE]

where $y_{o}=h(t,\,x_{o},\,\pi)$ and $\Theta\neq 0$, is a dynamically private version of (16). If $x^{\ast}(x_{o})=(L+\Theta)^{-1}\Theta x_{o}$ is the globally asymptotically stable equilibrium point of (16), then $x^{\ast}(x_{o})$ is a globally uniform attractor of (18).

Corollary 2

The masked system (18) is asymptotically autonomous with (16) as limit system. The $\omega$-limit set of (18) is given by $\{x^{\ast}(x_{o})\}=\{(L+\Theta)^{-1}\Theta x_{o}\}$ for each $x_{o}$.

Example 2

An example of $n=100$ agents is shown in Fig. 2. The introduction of $h(\cdot)$ scrambles the initial conditions, as expected, see panel (c) of Fig. 2. A level of privacy $\lambda=1$ is requested. Both $x(t)$ and $y(t)$ converge to the same $x^{\ast}=(L+\Theta)^{-1}\Theta x_{o}$, see panel (d) of Fig. 2, although neither now respects the rankings during the transient (i.e., unlike for (16), for (18) it is no longer true that $x_{i}(t_{1})<x_{j}(t_{1})$ $\Longrightarrow$ $x_{i}(t_{2})<x_{j}(t_{2})$ for all $t_{2}>t_{1}$).

5 Dynamically private average consensus

In the average consensus problem, $f(x)=-Lx$, with $L$ a weight-balanced Laplacian matrix: $L\textbf{1}=L^{T}\textbf{1}=0$, with $\textbf{1}=\begin{bmatrix}1&\ldots&1\end{bmatrix}^{T}\in\mathbb{R}^{n}$. When $L$ is irreducible, the equilibrium point is $x^{\ast}(x_{o})=(\textbf{1}^{T}x_{o}/n)\textbf{1}$. The system has a continuum of equilibria, described by ${\rm span}(\textbf{1})$, and each $x^{\ast}(x_{o})$ is globally asymptotically stable in ${\rm span}(\textbf{1})^{\perp}$, see [36].

Theorem 3

Consider the system

[TABLE]

where $L$ is an irreducible, weight-balanced Laplacian matrix, and denote $\eta=\textbf{1}^{T}x_{o}/n$ its average consensus value. Then $x^{\ast}=\eta\textbf{1}$ is a global uniform attractor on ${\rm span}(\textbf{1})^{\perp}$ for the masked system

[TABLE]

Furthermore, if Assumption 1 holds, then (20) is a dynamically private version of (19).

Also in this case our masked system is an asymptotically autonomous time-varying system.

Corollary 3

The masked system (20) is asymptotically autonomous with (19) as limit system. The $\omega$-limit set of (20) is given by $\{\eta\textbf{1}\}$ for each $x_{o}$.

Remark 4

Even if (19) has $x^{\ast}=\eta\textbf{1}$ as a globally asymptotically stable equilibrium point in ${\rm span}(\textbf{1})^{\perp}$, the masked system (6) does not have equilibria because of the extra inhomogeneous term in the right hand side, hence we cannot talk about stability of $\eta\textbf{1}$. Nevertheless, $x^{\ast}=\eta\textbf{1}$ remains a global attractor for all trajectories of the system in ${\rm span}(\textbf{1})^{\perp}$.

Remark 5

Since the evolution of the masked system (19) is restricted to the $n-1$ dimensional subspace ${\rm span}(\textbf{1})^{\perp}$, our masked consensus problem (as any exact privacy preserving consensus scheme) makes sense only when $n>2$. The case $n=2$ never satisfies Assumption 1 when $L$ is irreducible.

Example 3

In Fig. 3 a private consensus problem is run among $n=100$ agents. Both $x(t)$ and $y(t)$ converge to the same consensus value $\eta=\textbf{1}^{T}x(0)/n$, but the initial condition $y(0)$ does not reflect $x(0)$, not even when $x_{i}(0)$ is already near $\eta$ ($h(\cdot)$ does not preserve neighborhoods, see panel (c) of Fig. 3). The level of privacy measure imposed in this simulation is $\lambda=1$. Notice that $\textbf{1}^{T}x(t)/n$ is constant over $t$, while $\textbf{1}^{T}y(t)/n$ is not, i.e., the output mask hides also the conservation law. Notice further that a standard Lyapunov function used for consensus, like $V_{mm}(t)=\max_{i}(x_{i}(t))-\min_{i}(x_{i}(t))$, does not work in our privacy-preserving scheme (see panel (d) of Fig. 3), which reflects the fact that the system (20) is not asymptotically stable in ${\rm span}(\textbf{1})^{\perp}$. The convergence speed of the time-dependent part can be manipulated by selecting the factors $\sigma_{i}$ and $\delta_{i}$ appropriately.

6 Privacy for higher order systems: the case of pinned synchronization

When instead of a scalar variable, at each node we have a vector of variables $x_{i}\in\mathbb{R}^{\nu}$, $\nu>1$, then the definition of output mask can be straightforwardly extended by defining $h_{i}(t,x_{i},\pi_{i})$ as a $\nu$-dimensional diagonal map. For instance for the vanishing affine output mask, in place of (10) at each node we can use

[TABLE]

where $\Phi_{i}={\rm diag}(\phi_{i,1},\ldots,\phi_{i,\nu})$, $\Sigma_{i}={\rm diag}(\sigma_{i,1},\ldots,\sigma_{i,\nu})$, $\Delta_{i}={\rm diag}(\delta_{i,1},\ldots,\delta_{i,\nu})$, and $\gamma_{i}=\begin{bmatrix}\gamma_{i,1}&\ldots&\gamma_{i,\nu}\end{bmatrix}^{T}$. The formalism introduced in the paper extends unaltered.

We will now investigate privacy protection in a standard example of coordination of multivariable multiagent systems: synchronization via pinning control of identical nonlinear systems with diffusive couplings [10, 53, 54]. Other settings of multiagent coordination can be treated in an analogous way.

Consider a network of $n$ agents obeying the following set of coupled differential equations

[TABLE]

where $x_{i}\in\mathbb{R}^{\nu}$, $L=(\ell_{ij})$ is an irreducible Laplacian matrix, and $R$ is a symmetric positive definite matrix of inner couplings. The extra term in the first $k$ equations expresses the coupling with a pinned node ($p_{i}=$ pinning gain), acting as an exosystem for (21)-(22) and obeying to the law

[TABLE]

The system (23) can represent an equilibrium point, a periodic or a chaotic system [53]. Synchronization of (21)-(22) to the exosystem (23) corresponds to

[TABLE]

We need the following (standard) assumption:

Assumption 2

(Global Lipschitzianity of the drift)* $f\,:\,\mathbb{R}\to\mathbb{R}$ is such that*

[TABLE]

for some positive constant $q$.

Under Assumption 2, then a sufficient condition for global synchronization of (21)-(22) to (23) is given by the following matrix inequality

[TABLE]

where $\Xi={\rm diag}(\xi)$, with $\xi=(\xi_{1},\ldots,\xi_{n})$ the left eigenvector of $L$ relative to [math], and $P={\rm diag}(p_{1},\ldots,p_{k},0,\ldots,0)$, see [53] for more details.

Theorem 4

Under the Assumptions 1 and 2, if the solution $s(t)$ of (23) is bounded $\forall\,t\in[0,\,\infty)$, $L$ is irreducible, and $P$ is such that (25) holds, then the exosystem (23) is a global attractor for the trajectories of the dynamically private system:

[TABLE]

Remark 6

Notice that the masked system (26)-(28) is not asymptotically autonomous, as its limit system (21)-(22) is a function of the exosystem $s(t)$ which also constitutes the $\omega$-limit set of the system.

Example 4

Consider the case of an $f(\cdot)$ representing a three dimensional chaotic attractor (here the model presented in [54] is used). In Fig. 4 a system of $n=50$ coupled agents synchronize to an exosystem $s(t)$ obeying the same law. The privacy measure in this example is set to $\lambda=10$. The convergence speed can be tuned by changing the $\Sigma_{i}$ and $\Delta_{i}$ parameters of the masks.

7 Conclusions

The approach to privacy protection we have taken in this paper is exact and inspired by classical nonlinear systems techniques. While most of the assumptions under which it holds are fairly simple and reasonable (only the internal state of an agent and the parameters of its output mask must be kept private), the need to have non completely covering neighborhoods (Assumption 1) is instead restrictive, but difficult to dispense with without requiring some other form of restriction (for instance privacy of the vector fields themselves). Assumption 1 is key to guarantee the impossibility for an eavesdropper to identify a model of the system, and hence to set up an observer for $x_{o}$. Notice that a breaching of the privacy at one node does not compromise the other nodes.

From a system-theoretical perspective, the most interesting fact described in the paper is that privacy seems incompatible with a point being a fixed point of a dynamical system, as in that case if all agents happen to have initial conditions already on the fixed point, privacy is compromised (an agent will see the same stationary messages being exchanged among its neighboring nodes for all $t$). By extension of the same argument, approximate privacy (at any level of accuracy) does not seem to be compatible with stability. It is intriguing to investigate if concepts like $\epsilon$-differential privacy [11] can be rephrased in these more dynamical terms.

Several generalizations of our approach are possible. First of all an equivalent framework for discrete-time systems should be developed. Then it is easy to think of output masks that vanish in finite time rather than asymptotically. More complicated seems to be integrating the time dependence introduced by an output mask with a time-varying communication graph. Even more challenging is the case in which, instead of global exponential stability (perhaps on “slices” of the state space if there is a continuum of equilibria) of the unmasked system, this last has multiple isolated locally exponentially stable equilibria. In this case even a transient output mask may lead to tipping over from one basin of attraction to another, hence it should be used with care.

8 Acknowledgments

The author would like to thank Claudio De Persis for useful discussions on the topic of the paper and the anonymous reviewers for constructive criticisms. This paper is dedicated to the memory of the author’s father.

Appendix A Appendix

A.1 Auxiliary Lemmas

The following lemma is inspired by [33], Thm 2.1 and [44], Prop. 5, and provides us with a suitable comparison function to be used later in the paper.

Lemma 1

Consider the scalar system

[TABLE]

If $\alpha(v)\in\mathcal{K}_{\infty}^{2}$, $\beta\in\mathcal{KL}_{\infty}^{1,e}$ and $\zeta\in\mathcal{L}^{e}$, then the solutions of (29) are all prolongable to $\infty$ and bounded $\forall\;v_{o}\geq 0$ and $\forall t_{o}\geq 0$. Furthermore,

[TABLE]

Proof.

It follows from $\alpha(t)\geq 0$, $\beta(v,t)\geq 0$, $\zeta(t)\geq 0$ for all $v>0$ and $\alpha(0)=0$ that $\mathbb{R}_{+}$ is invariant for the system (29). If we can show that (29) remains bounded for all times, then (29) is also forward complete for all $v_{o}\geq 0$. Express $\alpha(v)\in\mathcal{K}_{\infty}^{2}$ as $\alpha(v)=av^{2}$, $\beta(v,t)\in\mathcal{KL}_{\infty}^{1,e}$ as $\beta(v,t)=bve^{-\delta_{1}t}$ and $\zeta\in\mathcal{L}^{e}$ as $\zeta(t)=ce^{-\delta_{2}t}$ for some $a,\,b,\,c>0$. Informally, boundedness follows from the fact that the globally exponentially stable “unperturbed” system $\dot{v}=-av^{2}$ has a higher order as $v\to\infty$ than the “perturbation” $bve^{-\delta_{1}t}+ce^{-\delta_{2}t}$. More in detail, in $\mathbb{R}_{+}$, for $v>1$ it is $v<v^{2}$, hence we can write

[TABLE]

or

[TABLE]

meaning that for $v>\max\left(1,\,\frac{be^{-\delta_{1}t_{o}}+ce^{-\delta_{2}t_{o}}}{a}\right)$ it is $\dot{v}<0$, $\forall\,t\geq t_{o}$, i.e., the solution of (29) remains bounded $\forall\,v_{o}\geq 0$ and $\forall\,t_{o}\geq 0$.

Furthermore, $\beta(v,t)$ and $\zeta(t)$ continuous, decreasing in $t$ with $\beta(v,t){\to 0}$ and $\zeta(t)\to 0$ as $t\to\infty$, imply also that for any $v_{o}>0$ there exists a $t_{1}\geq t_{o}$ such that $\forall\;t>t_{1}$ $\dot{v}(t)<0$. Together with $\mathbb{R}_{+}$-invariance, this implies that $\lim_{t\to\infty}v(t)=d\geq 0$. To show that it must be $d=0$, let us assume by contradiction that $d>0$. Then

[TABLE]

meaning that there exists a $t_{2}>t_{1}$ and a $k\in(0,\,1)$ such that

[TABLE]

Applying the mean value theorem, we then have that $\exists$ $\tau\in[t_{2},\,t]$ such that

[TABLE]

from which it follows

[TABLE]

which is a contradiction since $v\geq 0$.

With Lemma 1 in place, we can easily obtain the following sufficient condition for global convergence to the origin of a time-varying system in which the Lyapunov function has time derivative that is sign indefinite but bounded above by $\mathcal{KL}_{\infty}^{1,e}$ and $\mathcal{L}^{e}$ functions, i.e., by terms growing linearly in the norm of the state and decaying exponentially in time.

Lemma 2

Assume that in the time-varying system (1) $g\,:\,\mathbb{R}_{+}\times\mathbb{R}^{n}\to\mathbb{R}^{n}$ is such that the solution of (1) exists unique in $[t_{o},\infty)$ $\forall\;x_{o}\in\mathbb{R}^{n}$ and $\forall\;t_{o}\geq 0$. If there exists a continuously differentiable function $V(t,\,x)\,:\,\mathbb{R}_{+}\times\mathbb{R}^{n}\to\mathbb{R}$, three $\alpha_{1}$, $\alpha_{2}$, $\alpha_{3}\in\mathcal{K}_{\infty}^{2}$, $\beta\in\mathcal{KL}_{\infty}^{1,e}$ and $\zeta\in\mathcal{L}^{e}$ such that

[TABLE]

$\forall$* $t\geq t_{o}$, $t_{o}\geq 0$ and $x_{o}\in\mathbb{R}^{n}$, then any solution of (1) converges to [math] uniformly in $t_{o}$ as $t\to\infty$.*

Proof.

The right-hand side of (31) has the same structure as that of (29), meaning we can apply the comparison lemma, using (29) with initial condition $v(t_{o})=V(t_{o},x_{o})$. Denoting $v(t)$ the corresponding solution, it is then

[TABLE]

From Lemma 1 and (32), it follows that for all $x_{o}$ it must be $\lim_{t\to\infty}V(t,x(t))=0$ for any $t_{o}\geq 0$, hence from (30) $\lim_{t\to\infty}\alpha_{1}(x(t))=0$ or $\lim_{t\to\infty}x(t)=0$.

Remark 7

The sufficient conditions of Lemma 1 (and hence of Lemma 2) can be rendered more general using for instance the properties of input-to-state stability [47], or of cascade nonlinear systems [38].

Lemma 3

For the output mask (11), it holds:

[TABLE]

where $k=\|I+\Phi\|$ and $\zeta(t)=\|e^{-\Delta t}\|\in\mathcal{L}^{e}$.

Proof.

The inverse of (11) is

[TABLE]

Notice that $\|I+\Phi e^{-\Sigma t}\|\leq\|I+\Phi\|=k$. We have from (11) and from the definition of $\zeta(t)$ that

[TABLE]

and from (34),

[TABLE]

The bounds (33) follow combining these two inequalities.

A.2 Proof of Proposition 1

For (8) it is $\rho_{i}(x_{o,i})=|\gamma_{i}|$, hence it is enough that each agent chooses the parameter $\gamma_{i}$ such that $|\gamma_{i}|>\lambda$, independently of $x_{o,i}$. For (9), $\rho_{i}(x_{o,i})=|(c_{i}-1)x_{o,i}+\gamma_{i}|>\lambda$ is satisfied for an infinite number of parameter pairs $(c_{i},\,\gamma_{i})$ with $c_{i}>1$ and $\gamma_{i}\neq 0$. When this inequality is satisfied for all agents then $\rho(x_{o})>\lambda$. Similarly, for (10) there exist infinitely many parameter pairs $(\phi_{i},\,\gamma_{i})$ with $\phi_{i}>0$ and $\gamma_{i}\neq 0$ satisfying $\rho_{i}(x_{o,i})=|\phi_{i}x_{o,i}+(1+\phi_{i})\gamma_{i}|>\lambda$.

A.3 Proof of Proposition 2

Proof.

Consider an eavesdropping agent $j$ trying to discern the initial condition of agent $i$, i.e., trying to estimate $x_{o,i}$ based on K1 and K2. This requires to compute the state of $i$ from the available outputs. From (12), there are two possible ways to proceed. The first is to proceed “statically” by inverting $h_{i}(\cdot)$ in (12b), and the second to proceed dynamically using both (12a) and (12b). Concerning the first possibility, from (12b) computing $x_{i}(t,x_{o,i})$ from $y_{i}(t,x_{o,i})$ requires to invert the masked map $h_{i}(\cdot)$ for each $t$. From U1, however, this inversion is not possible for agent $j$, because $h_{i}(\cdot)$ is a privacy mask unknown to agent $j$. Concerning the second possibility, there are two possible options: the first is to use (12a) and (12b) to set up a system identification problem for $h_{i}(\cdot)$ and $\pi_{i}$. However, from Assumption 1, only a proper subset $\mathcal{M}_{ij}=\{\mathcal{N}_{i}\cup\{i\}\}\cap\{\mathcal{N}_{j}\cup\{j\}\}$ of all output trajectories entering into $f_{i}(\cdot)$ (i.e., $\{\mathcal{N}_{i}\cup\{i\}\}$) is available to agent $j$. Combining this with the essential neighborhood assumption (4), we obtain that agent $j$ cannot correctly compute the right hand side of (12a), hence a system identification problem for (12) cannot be solved correctly. The second dynamical option is instead to consider the formal solution of (12a)

[TABLE]

Since $y^{\ast}=\lim_{t\to\infty}y(t)=\lim_{t\to\infty}x(t)=x^{\ast}$, if $i\in\mathcal{N}_{j}$, when $t\to\infty$, $x_{i}^{\ast}$ is available to $j$, hence $x_{o,i}$ can be expressed as

[TABLE]

Also in this case, however, Assumption 1 combined with the essential neighborhood assumption (4) implies that agent $j$ cannot correctly estimate the integral in (35), as $\mathcal{M}_{ij}\nsubseteq\{\mathcal{N}_{i}\cup\{i\}\}$. In summary, since neither static nor dynamical methods for estimating $x_{o,i}$ can be applied, we can conclude that the initial condition $x_{o,i}$ is indiscernible for agent $j$. Since Assumption 1 is valid for all agents, we can also conclude that $x_{o}$ is indiscernible, and therefore that (6) is a dynamically private version of (2).

A.4 Proof of Proposition 3

The right hand side of the dynamics in (6) is autonomous. Assume there exists $y^{\ast}$ such that $f(y^{\ast})=0$. Since, from P4 of Definition 3, $h(\cdot)$ is invertible in $x$ for each $t$, by the implicit function theorem, there exists an $x^{\ast}(t)$ such that $y^{\ast}=h(t,x^{\ast}(t),\pi)$. If $x^{\ast}(t)$ is time-varying, then it is not an equilibrium point for (6). If instead $x^{\ast}$ is time-invariant, then, from $\lim_{t\to\infty}y(t)=\lim_{t\to\infty}x(t)$, it must be $x^{\ast}=y^{\ast}$. But then, choosing $x(0)=x^{\ast}$, it is $y^{\ast}=h(0,x^{\ast},\pi)=x^{\ast}$, i.e., P2 of Definition 3 is violated, hence also this case cannot happen in a privacy mask.

As for the second part, we need to show that $f(h(t,x,\pi))\to f(x)$ as $t\to\infty$ uniformly on compacts of $\mathbb{R}^{n}$ [6]. From P5 and $h\in C^{1}$, there exists an increasing, diverging sequence $\{t_{k}\}$ for which $h_{i}(t_{k},x_{i},\pi_{i})\to x_{i}$ as $t_{k}\to\infty$, i.e., pointwise convergence holds. In particular, for any $\epsilon>0$, from pointwise convergence, there exists a $\nu_{o}(x_{i})$ such that, for all $\nu>\nu_{o}$, $|h_{i}(t_{\nu},x_{i},\pi_{i})-x_{i}|<\epsilon/2$. Pick two indexes $\nu_{1}=\nu_{1}(x_{i})$, $\nu_{2}=\nu_{2}(x_{i})$ such that $\nu_{m}>\nu_{o}$, $m=1,\,2$. Then $|h_{i}(t_{\nu_{1}},x_{i},\pi_{i})-h_{i}(t_{\nu_{2}},x_{i},\pi_{i})|\leq|h_{i}(t_{\nu_{1}},x_{i},\pi_{i})-x_{i}|+|h_{i}(t_{\nu_{2}},x_{i},\pi_{i})-x_{i}|\leq\epsilon/2+\epsilon/2$. Selecting $\nu_{s}=\sup_{x_{i}\in\mathcal{X}_{i}}\left\{\nu_{m}(x_{i}),\;m=1,\,2\right\}$, then the Cauchy condition for uniform convergence applies and we have for any integer $\mu$

[TABLE]

Hence, for a certain subsequence $\{t_{\nu}\}$ of $\{t_{k}\}$ it is ${\rm sup}_{x_{i}\in\mathcal{X}_{i}}\left|h_{i}(t_{\nu},x_{i},\pi_{i})-x_{i}\right|\to 0$ as $k\to\infty$, meaning that for $h_{i}$ convergence is uniform on compacts. Since $f_{i}$ is Lipschitz continuous, it is uniformly continuous and bounded on compacts. Hence Lemma 1 of [26] holds, and by a reasoning identical to the one above, if $\mathcal{X}$ is a compact of $\mathbb{R}^{n}$ we have:

[TABLE]

The argument holds independently for any component $f_{i}$. Asymptotic time-independence and uniform convergence on compacts to $f(x)$ follow consequently.

A.5 Proof of Theorem 1

By a standard converse theorem (e.g. Thm 4.14 of [22]), global exponential stability of (2) with $f$ globally Lipschitz implies $\exists$ a $C^{1}$ positive definite and radially unbounded Lyapunov function $V\,:\,\mathbb{R}^{n}\to\mathbb{R}_{+}$ and constants $b_{i}>0$, $i=1,\ldots,4$, such that $\forall\,x\in\mathbb{R}^{n}$

[TABLE]

With $y=C(x+e^{-\Delta t}\gamma)$, the system (6) can be rewritten as

[TABLE]

Considering $V$ evaluated in $y$, and computing its derivative along the trajectories of (39), we get:

[TABLE]

Defining $k_{1}=\|C\|>0$, since (37) is valid everywhere, it is

[TABLE]

for some $\alpha\in\mathcal{K}_{\infty}^{2}$, while the second term of (40) can be rewritten as

[TABLE]

For each $t$, $c_{i}\delta_{i}\gamma_{i}e^{-\delta_{i}t}\leq\zeta_{1}(t)\triangleq\max_{i}\left(c_{i}\delta_{i}|\gamma_{i}|\right)\max_{i}\left(e^{-\delta_{i}t}\right)\in\mathcal{L}^{e}$. Since

[TABLE]

from (38), Lemma 3 (where we impose $\sigma_{i}=0$ and $c_{i}=1+\phi_{i}$) and $\|C^{-1}\|\leq 1$, it is

[TABLE]

for some $\zeta_{2}\in\mathcal{L}^{e}$. Hence, for some constant $k_{2}>0$,

[TABLE]

with $\beta\in\mathcal{KL}_{\infty}^{1,e}$ and $\zeta_{3}\in\mathcal{L}^{e}$. Therefore,

[TABLE]

which has the same structure of (31), meaning that we can apply Lemma 2 and conclude that the system (2) is uniformly globally attracted to $x^{\ast}=0$. Since (13) is a privacy mask, $\lim_{t\to\infty}y(t)=0$ and Assumption 1 holds, from Proposition 2, (6) is a dynamically private version of (2).

A.6 Proof of Corollary 1

Asymptotic autonomy of (39) is shown using an argument identical to that of the proof of Proposition 3. Convergence to the limit system $\dot{y}=Cf(y)$ and hence (14) follows consequently. From expression (39) it is also clear that, for all $y_{o}=h(0,x_{o},\pi)$, $\Omega_{y_{o}}=\{0\}$, hence so it is for (6).

A.7 Proof of Theorem 2

When $\Theta\neq 0$, $-(L+\Theta)$ is Hurwitz, as can be easily deduced from e.g. [10]. In the $z=x-x^{\ast}$ basis, for the unmasked system (16) a quadratic Lyapunov function can be used: $V(z)=z^{T}Pz$, where $P=P^{T}>0$ is the solution of the Lyapunov equation

[TABLE]

in correspondence of a given $Q=Q^{T}>0$. The masked system (18) can be rewritten as

[TABLE]

or, in $z$, after easy manipulations,

[TABLE]

where $[\,\cdot\,,\,\cdot\,]$ is the matrix commutator. Notice that for this term we have $\|B(t)\|\leq\zeta_{1}(t)\in\mathcal{L}^{e}$. Inserting $\dot{z}$ in $\dot{V}$:

[TABLE]

Looking at the terms of (42):

[TABLE]

where $\alpha_{1}\in\mathcal{K}_{\infty}^{2}$, $\alpha_{2},\,\alpha_{3}\in\mathcal{K}_{\infty}^{1}$ and $\zeta_{i}\in\mathcal{L}^{e}$, meaning that $\beta_{i}(\|z\|,t)=\alpha_{i}(\|z\|)\zeta_{i}(t)\in\mathcal{KL}_{\infty}^{1,e}$, $i=2,\,3$. Therefore, overall we can write

[TABLE]

where $\beta(\|z\|,t)=\max_{j=2,3}\alpha_{j}(\|z\|)\max_{j=2,3}\zeta_{j}(t)\in\mathcal{KL}_{\infty}^{1,e}.$ Since $V$ is quadratic, positive definite, radially unbounded and vanishing in $z=0$, there exists two class $\mathcal{K}_{\infty}^{2}$ functions $\alpha_{4}$ and $\alpha_{5}$ such that

[TABLE]

Hence we can apply Lemma 2 and obtain $\lim_{t\to\infty}z(t)=0$. In the original variables $x$, this implies $\lim_{t\to\infty}x(t)=x^{\ast}(x_{o})$ for all $x_{o}$. Convergence of $x$ to $x^{\ast}(x_{o})$ is uniform in $t$ because $V$ does not depend on time.

A.8 Proof of Corollary 2

The first part follows from Proposition 3 and the second from $x^{\ast}(x_{o})$ being a uniform attractor for each $x_{o}$.

A.9 Proof of Theorem 3

Notice first that the system (20) can be written as

[TABLE]

from which it is clear that the system (20) cannot have equilibrium points. It is also clear from (47) that $\textbf{1}^{T}\dot{x}=0$ i.e., also (20) obeys to the conservation law $\textbf{1}^{T}x(t)=\textbf{1}^{T}x_{o}=\eta\textbf{1}$. As in the standard consensus problem [36], we can therefore work on the $n-1$ dimensional projection subspace ${\rm span}(\textbf{1})^{\perp}$ and consider the time-varying Lyapunov function for the “displacement vector” $x-\eta\textbf{1}\in{\rm span}(\textbf{1})^{\perp}$:

[TABLE]

From now on we assume that all calculations are restricted to ${\rm span}(\textbf{1})^{\perp}$. The derivative of $V$ along the solutions of (20) is

[TABLE]

Since $\phi_{i}>0$, it is $1+\phi_{i}e^{-\sigma_{i}t}\geq 1$ $\forall\,t\geq 0$, and $I+\Phi e^{-\Sigma t}$ is a positive definite diagonal matrix, for the first term of (48) we have

[TABLE]

for some function $\alpha_{1}\in\mathcal{K}_{\infty}^{2}$. The second term of (48) is linear in $\|x-\eta\textbf{1}\|$, and from $L\textbf{1}=L^{T}\textbf{1}=0$, we have

[TABLE]

for some function $\beta_{1}\in\mathcal{KL}_{\infty}^{1,e}$. Similarly, for the third term of (48),

[TABLE]

for some $\beta_{2}\in\mathcal{KL}_{\infty}^{1,e}$. Finally, the fourth term of (48) is

[TABLE]

for some $\alpha_{2}\in\mathcal{KL}_{\infty}^{2,e}$, i.e., it is positive definite for all finite $t$, and vanishes as $t\to\infty$. hence there exists a $\alpha\in\mathcal{K}_{\infty}^{2}$ such that

[TABLE]

Denote $\beta(\|x-\eta\textbf{1}\|,t)\in\mathcal{KL}_{\infty}^{1,e}$ a proper majorization of $\beta_{j}(\|x-\eta\textbf{1}\|,t)$, $j=1,\,2$. Since, for all $t$, $V$ is quadratic, positive definite, radially unbounded and vanishing in $x=\eta\textbf{1}$, there exists two class $\mathcal{K}_{\infty}^{2}$ functions $\alpha_{3}$ and $\alpha_{4}$ such that

[TABLE]

Also in this case we can apply the comparison lemma, using (29) with initial condition $v(0)=V(0,x_{o})$, where $x_{o}$ such that $\textbf{1}^{T}x_{o}/n=\eta$. From Lemma 1, it follows that it must be $\lim_{t\to\infty}V(t,x(t))=0$ for all $x_{o}$ such that $\textbf{1}^{T}x_{o}/n=\eta$, hence from (49) $\lim_{t\to\infty}\alpha_{3}(\|x-\eta\textbf{1}\|)=0$ or $\lim_{t\to\infty}x(t)=\eta\textbf{1}$ for all $x_{o}$ such that $\textbf{1}^{T}x_{o}/n=\eta$. Since $h(t,x,\pi)=\left(I+\Phi e^{-\Sigma t}\right)\left(x+e^{-\Delta t}\gamma\right)$ is a privacy mask $\lim_{t\to\infty}y(t)=\eta\textbf{1}$ and Assumption 1 holds, from Proposition 2, (20) is a dynamically private version of (19).

A.10 Proof of Corollary 3

Same as proof of Corollary 2.

A.11 Proof of Theorem 4

Notice first that (24) implies the following one-sided global Lipschitz condition used in [53]:

[TABLE]

Denoting $e_{i}(t)=x_{i}(t)-s(t)$ the error of the $i$-th system from the desired trajectory, and using (28), then (26) can be written in terms of $e_{i}$ as

[TABLE]

Denote $e=\begin{bmatrix}e_{1}^{T}&\ldots e_{n}^{T}\end{bmatrix}^{T}$ and, for brevity, $\Psi_{i}(t)=I+\Phi_{i}e^{-\Sigma_{i}t}$. A Lyapunov function, derived by that used in the standard pinned synchronization problem [53], is the following:

[TABLE]

Since, for all $t$, $V(t,e)$ is quadratic, positive definite, vanishing at $e=0$, and radially unbounded, there exist two functions $\alpha_{1},\,\alpha_{2}\in\mathcal{K}_{\infty}^{2}$ such that

[TABLE]

For its derivative along the trajectories of (26)-(28) it is:

[TABLE]

Of the eight terms on the right hand side, the first is the most complicated and will be treated last. Three other are quadratic in $\|e\|$ and can be written as in [53], using Kronecker products:

[TABLE]

where $\Psi={\rm diag}(\Psi_{1},\ldots,\Psi_{n})$, $\Sigma={\rm diag}(\Sigma_{1},\ldots,\Sigma_{n})$ and $\Phi={\rm diag}(\Phi_{1},\ldots,\Phi_{n})$. The remaining four are all linear in $\|e\|$ and decaying exponentially in $t$, and can be majorized in the following way

[TABLE]

with $\zeta_{1}(t)=\max_{i,j}\!\!\left\{\left\|\Psi_{i}(0)\right\|\!\cdot\!\left\|\Phi_{j}\right\|\!\cdot\!\|s(t)\|_{\infty}\right\}\max_{j}\left\{e^{-\Sigma_{j}T}\right\}\in\mathcal{L}^{e}$ ($s(t)$ is bounded for all $t$), $k_{1}>0$, and $\beta_{1}\in\mathcal{KL}_{\infty}^{1,e}$;

[TABLE]

$\zeta_{2}(t)=\max_{j}\left\{\left\|\Psi_{i}(0)\right\|\!\cdot\!\left\|\Psi_{j}(0)\gamma_{j}\right\|\right\}\max_{j}\left\{e^{-\Delta_{j}t}\right\}\in\mathcal{L}^{e}$, $k_{2}>0$, and $\beta_{2}\in\mathcal{KL}_{\infty}^{1,e}$;

[TABLE]

$k_{3}>0$, $\beta_{3}\in\mathcal{KL}_{\infty}^{1,e}$;

[TABLE]

$k_{4}>0$, $\beta_{4}\in\mathcal{KL}_{\infty}^{1,e}$. Finally for (51a), from (34),

[TABLE]

where $F_{i}(t)$ is diagonal, positive definite, $\|F_{i}(t)\|\leq 1$, and $\lim_{t\to\infty}F_{i}(t)=I$. Hence

[TABLE]

where for the first term we have used the one-sided Lipschitz condition

[TABLE]

which follows from (50) and the equivalence of norms, and for the second term the fact that, from (24), it depends linearly from $\|y-\bar{s}\|$ and it decays exponentially to [math] as $t\to\infty$, meaning that $\beta_{5}\in\mathcal{KL}_{\infty}^{1,e}$ ($\bar{s}$ is the vector of $n$ identical copies of $s$). Furthermore, since, from Lemma 3, $\|y-\bar{s}\|\leq k\|e\|+\zeta_{4}(t)$ for some $\zeta_{4}\in\mathcal{L}^{e}$ and $k>1$, it is $\beta_{5}(\|y-\bar{s}\|,\,t)\leq\beta_{6}(\|e\|,\,t)+\zeta_{5}(t)$ with $\beta_{6}\in\mathcal{KL}_{\infty}^{1,e}$ and $\zeta_{5}\in\mathcal{L}^{e}$. Inserting

[TABLE]

and expanding, one gets a term quadratic in $\|e\|$,

[TABLE]

plus several other terms of first or zero order in $\|e\|$, all vanishing exponentially fast in $t$. As long as $s(t)$ is bounded, using arguments identical to those above, we can therefore write

[TABLE]

with $\beta_{7}\in\mathcal{KL}_{\infty}^{1,e}$ and $\zeta_{6}\in\mathcal{L}^{e}$. Putting together all terms quadratic in $\|e\|$, since $\Psi(t)$ is diagonal positive definite and $\Sigma\Phi e^{-\Sigma t}\Xi\otimes I$ is positive definite for all $t$, it follows from (25) that there exists $\alpha_{3}\in\mathcal{K}_{\infty}^{2}$ such that

[TABLE]

Hence

[TABLE]

where $\beta(\|e\|,t)\in\mathcal{KL}_{\infty}^{1,e}$ majorizes $\beta_{j}(\|e\|,t)$, $j=1,\ldots,7$, and $\zeta(t)\in\mathcal{L}^{e}$ majorizes $\zeta_{j}(t)$, $j=1,\ldots,6$, meaning that we can apply the comparison lemma (Lemma 1), using (29) with initial condition $v(0)=V(0,e(0))$, and the result follows.