Intrinsic Stability: Global Stability of Dynamical Networks and Switched Systems Resilient to any Type of Time-Delays

TL;DR

This paper demonstrates that intrinsically stable dynamical networks and switched systems remain stable despite any form of time-varying delays, enhancing robustness in real-world and computational networks.

Contribution

It proves that intrinsic stability ensures global stability under all types of time-delays, including stochastic and periodic, and shows this property is computationally efficient to verify.

Findings

Intrinsic stability guarantees stability with any time-varying delays.

The asymptotic state is exponentially independent of initial conditions.

The approach improves existing stability results and is computationally inexpensive.

Abstract

In real-world networks the interactions between network elements are inherently time-delayed. These time-delays can not only slow the network but can have a destabilizing effect on the network's dynamics leading to poor performance. The same is true in computational networks used for machine learning etc. where time-delays increase the network's memory but can degrade the network's ability to be trained. However, not all networks can be destabilized by time-delays. Previously, it has been shown that if a network or high-dimensional dynamical system is intrinsically stable, which is a stronger form of the standard notion of global stability, then it maintains its stability when constant time-delays are introduced into the system. Here we show that intrinsically stable systems, including intrinsically stable networks and a broad class of switched systems, i.e. systems whose mapping is time-dependent, remain stable in the presence of any type of time-varying time-delays whether these delays are periodic, stochastic, or otherwise. We apply these results to a number of well-studied systems to demonstrate that the notion of intrinsic stability is both computationally inexpensive, relative to other methods, and can be used to improve on some of the best known stability results. We also show that the asymptotic state of an intrinsically stable switched system is exponentially independent of the system's initial conditions.

Equations

$$d_{max}(\mathbf{x},\mathbf{y})=\underset{i}{\max }{d}_i(x_i,y_i)\text{ where }\mathbf{x},\mathbf{y}\in X\text{ and }{x}_i,y_i\in X_i.$$

$$F_i=F_i(x_1,x_2,\dots ,x_n)\text{in which }{x}_j\in X_j\text{for }j=1,2,\dots ,n$$

$$C_i(\mathbf{x})=(1-ϵ)x_i+\sum _{j=1}^n{W}_{ij}\sigma (x_j)+c_{i}1\le i\le n,$$

$$\left|\sigma (x)-\sigma (y)\right|\le K\left|x-y\right|$$

$$d_i(F_i(\mathbf{x}),F_i(\mathbf{y}))\le \sum _{j=1}^n{a}_{ij}{d}_j(x_j,y_j)\text{for all }\mathbf{x},\mathbf{y}\in X.$$

$$a_{ij}=\underset{x\in X}{\sup }\left|\frac{\partial F_i}{\partial x_j}(\mathbf{x})\right|$$

$$a_{ij}=\{\begin{array}{ll}\left|1-ϵ\right|+K\left|W_{ii}\right|& \text{if }i=j\\ K\left|W_{ij}\right|& \text{otherwise.}\end{array}$$

$$\rho (A)=\underset{\lambda \in \sigma (A)}{\max }|\lambda |$$

$$\rho (A)=\left|1-ϵ\right|+K\rho (\left|W\right|)$$

$$X_L=\underset{\ell =0}{\overset{L}{⨁}}\underset{i=1}{\overset{n}{⨁}}{X}_{i,\ell }\text{ where }{X}_{i,\ell }=X_i\text{ for }1\le i\le n\text{ and }0\le \ell \le L.$$

$$(F_D{)}_{i,\ell +1}\text{mathchar}58X_{i,\ell }\to X_{i,\ell +1}\text{ given by the identity map }(F_D{)}_{i,\ell +1}(x_{i,\ell })=x_{i,\ell }\text{ for }0\le \ell \le L-1$$

$$(F_D{)}_{i,0}\text{mathchar}58\underset{j=1}{\overset{n}{⨁}}{X}_{j,d_{ij}}\to X_{i,0}\text{ given by }(F_D{)}_{i,0}=F_i(x_{1,d_{i1}},x_{2,d_{i2}},\dots ,x_{n,d_{in}})$$

$$\mathbf{x}=[x_{1,0},x_{2,0},\dots ,x_{n,0},x_{1,1},x_{2,1},\dots ,x_{n,L}{]}^T$$

$$C\bigg{(}\begin{bmatrix}x_{1}\\ x_{2}\end{bmatrix}\bigg{)}=\begin{bmatrix}C_{1}(x_{1},x_{2})\\ C_{2}(x_{1},x_{2})\end{bmatrix}=\begin{bmatrix}(1-\epsilon)x_{1}+W_{11}\phi(x_{1})+W_{12}\phi(x_{2})+c_{1}\\ (1-\epsilon)x_{2}+W_{21}\phi(x_{1})+W_{22}\phi(x_{2})+c_{2}\end{bmatrix},$$

$$D=\left[\begin{array}{cc}1& 2\\ 1& 3\end{array}\right],$$

$$C_D\left(\left[\begin{array}{c}{x}_{1,0}\\ x_{2,0}\\ x_{1,1}\\ x_{2,1}\\ x_{1,2}\\ x_{2,2}\\ x_{1,3}\\ x_{2,3}\end{array}\right]\right)=\left[\begin{array}{c}{C}_1(x_{1,1},x_{2,2})\\ C_2(x_{1,1},x_{2,3})\\ x_{1,1}\\ x_{2,1}\\ x_{1,2}\\ x_{2,2}\\ x_{1,3}\\ x_{2,3}\end{array}\right]=\left[\begin{array}{c}(1-ϵ)x_{1,1}+W_{11}\varphi (x_{1,1})+W_{12}\varphi (x_{2,2})+c_1\\ (1-ϵ)x_{2,3}+W_{21}\varphi (x_{1,1})+W_{22}\varphi (x_{2,3})+c_2\\ x_{1,0}\\ x_{2,0}\\ x_{1,1}\\ x_{2,1}\\ x_{1,2}\\ x_{2,2}\end{array}\right]$$

$$W=\left[\begin{array}{cc}0& -\frac{3}{4}\\ \frac{3}{4}& 0\end{array}\right],$$

$$\rho (A)=\left|1-ϵ\right|+K\rho (\left|W\right|)=1.35>1.$$

$$E_L(\mathbf{x})=\left[\begin{array}{c}{\mathbf{x}}_0\\ {\mathbf{x}}_1\\ ⋮\\ {\mathbf{x}}_L\end{array}\right]\text{ where }{\mathbf{x}}_{\ell }=\mathbf{x}\text{ for }0\le \ell \le L.$$

$${\mathcal{F}}^k(\mathbf{x})=F^{(k)}\circ \dots \circ F^{(2)}\circ F^{(1)}(\mathbf{x})\text{ for }\mathbf{x}\in X.$$

$$P(\mathbf{x})=\left[\begin{array}{cc}ϵ& 1\\ 0& ϵ\end{array}\right]\mathbf{x}\text{ and }P(\mathbf{x})=\left[\begin{array}{cc}ϵ& 0\\ 1& ϵ\end{array}\right]\mathbf{x}\text{ for small }ϵ<<1.$$

$$U(\mathbf{x})=\left[\begin{array}{cc}{ϵ}^2& ϵ\\ ϵ& 1+{ϵ}^2\end{array}\right]\text{ with }\rho (U)=\frac{1}{2}(1+2{ϵ}^2+\sqrt{1+4{ϵ}^2})>1.$$

$$||{\mathbf{z}}^k||\le C(\overline{\rho }{)}^k.$$

$$d_{max}({\mathcal{F}}^k({\mathbf{x}}^0),{\mathcal{F}}^k({\mathbf{y}}^0))\le C\overline{\rho }(S{)}^k.$$

$$G\bigg{(}\begin{bmatrix}x_{1}\\ x_{2}\end{bmatrix}\bigg{)}=\begin{bmatrix}(1-\epsilon_{1})x_{1}-\frac{3}{4}\phi(x_{2})+c_{1}\\ (1-\epsilon_{1})x_{2}+\frac{3}{4}\phi(x_{1})+c_{2}\end{bmatrix}\ \ \text{and}\ \ H\bigg{(}\begin{bmatrix}x_{1}\\ x_{2}\end{bmatrix}\bigg{)}=\begin{bmatrix}(1-\epsilon_{2})x_{1}+\frac{1}{4}\phi(x_{2})+d_{1}\\ (1-\epsilon_{2})x_{2}+\frac{1}{4}\phi(x_{1})+d_{2}\end{bmatrix},$$

$$S=\left\{\left[\begin{array}{cc}\frac{1}{5}& \frac{3}{4}\\ \frac{3}{4}& \frac{1}{5}\end{array}\right],\left[\begin{array}{cc}\frac{7}{10}& \frac{1}{4}\\ \frac{1}{4}& \frac{7}{10}\end{array}\right]\right\}.$$

$$\underset{k\to \infty }{\lim }{d}_{max}({\mathcal{F}}^k({\mathbf{x}}^0),{\mathcal{F}}^k({\mathbf{y}}^0))=0$$

$$M_d=\{F_D|D\in {\mathbb{N}}^{n\times n}\text{ with }\underset{ij}{\max }{d}_{ij}\le L\}$$

$$A_L=\left[\begin{array}{cc}{0}_{n\times nL}& A\\ {\mathbf{I}}_{nL\times nL}& 0_{nL\times n}\end{array}\right].$$

$$D^{(k)}=\left[\begin{array}{cc}k\mathrm{mod}5& k\mathrm{mod}6\\ k\mathrm{mod}6& k\mathrm{mod}5\end{array}\right].$$

Full text

1 Introduction

The study of networks deals with understanding the properties of systems of interacting elements. In the social sciences these elements are typically individuals whose social interactions create networks such as Facebook and Twitter. In the biological sciences networks range from metabolic networks of single-cell organisms to the neuronal networks of the brain to the larger physiological network of the organs within the body. Networks such as citation networks and other well-studied networks such as the World Wide Web belong to what are referred to as information networks. In the technological sciences examples of networks include the internet, power grids, and transportation networks. (For an overview of these different types of networks see [1].)

These real-world networks are dynamic in that both their topology, which is the network’s structure of interactions, and the state of the network are time dependent. Here our focus is on the changing state of the network, which is often referred to as the dynamics on the network. The state of the network is the collective states of the network elements. The fundamental concept in a dynamical network is that the state of a given element depends on the dynamics of its neighbors, which are the other network elements that directly interact with this element.

Here we refer to the emergent behavior of these interacting elements, which is the changing state of these network elements, as the network’s dynamics. The network’s dynamics can be periodic, as is found in many biological networks [2], synchronizing, which is the desired condition for transmitting power over large distance in power grids [3], and stable or multistable dynamics such as is found in gene-regulatory networks [4], etc.

In real-world networks the interactions between network elements are inherently time-delayed. This comes from the fact that network elements are spatially separated, that information and other quantities can only be processed and transmitted at finite speeds, and that these quantities can be slowed by network traffic [5]. These time-delays not only slow the network, leading to poor performance, but can have a destabilizing effect on the network’s dynamics, which can lead to network failure [6, 7]. The same is true in computational networks used for machine learning etc. where time-delays can be used to increase the network’s ability to detect long term temporal dependencies [8] but at the cost of potentially degrading the ability to train the network.

Not all networks can be destabilized by time-delays. In [9] the notion of intrinsic stability was introduced, which is a stronger form of the standard notion of stability, i.e. a system in which there is a globally attracting fixed point (see [10] for more details). If a network is intrinsically stable the authors showed that constant-type time-delays, which are delays that do not vary in time, have no effect on the network’s stability. As was shown in [9] an advantage of the intrinsic stability method over other methods such as Lyapunov-type methods, Linear Matrix Inequalities (LMI), and Semi-Definite Programming (SPD) is that determining whether a network is intrinsically stable can be done with respect to the lower-dimensional undelayed network and does not require the creation of special functions or the use of interior point methods, etc. What is required is finding the spectral radius of the network’s Lipschitz matrix, which for large network’s can be done efficiently by use of the power method [11].

In many situations, however, the delays networks or, or more generally high-dimensional systems, experience are not constant-type time-delays. Time delays can be periodic, such as the daily/annual cycles in population models [12], or even stochastic as in traffic models [13]. It is worth noting that these time-varying time-delays are more complicated than constant-type time delays and as such the theory of systems with time-varying time-delays is less developed than the theory of systems with constant time-delays, which in turn is less developed than the theory of systems without delays.

The main goal of this paper is to further develop the theory describing the stability of dynamical systems that experience time delays. Here we show that intrinsically stable systems, including intrinsically stable networks and a broad class of switched systems, i.e. systems whose mapping is time-dependent, remain stable in the presence of any time-varying time-delays whether these delays are periodic, stochastic, or otherwise (see Main Results 2 and 3). We apply these results to a number of well-studied systems to demonstrate that the notion of intrinsic stability is both computationally inexpensive, relative to other methods, and can be used to improve on some of the best known results. (See for instance Example 4.7, compared to results found in [14, 15, 16, 17, 18].) We also show that the asymptotic state of intrinsically stable switched systems is independent of the system’s initial conditions (see Main Result 1). This allows us to show that the globally attracting state of any intrinsically stable network and any time-delayed version of the network are identical (see Proposition 1).

The main results of this paper are demonstrated using Cohen-Grossberg Neural (CGN) Networks, which are dynamical networks whose stability is often studied in the presence of constant-type and time-varying time-delays [19, 20]. It is worth noting that the results of this paper justify the modeling of dynamical networks and switched systems without formally including delays in the model if it is known that the system is intrinsically stable. The reason is that although delays do change the specific details the system’s dynamics, if the system is intrinsically stable it will have the same qualitative dynamics and same asymptotic state whether or not its delays are included (see Main Results 1, 2, and 3)). The advantage is that determining or correctly anticipating what delays a network will experience can be quite complicated for most any real-world system. Hence, this theory also has implications to system design as an intrinsically stable network will not be destabilized by any type of unexpected delays.

As this theory is in many ways different from the standard theory of stable dynamical systems, much effort has gone into, first, making this theory understandable and, second, emphasizing the computational and algorithmic advantages of this method. To this end examples are given throughout this paper describing each result, its usefulness, and how these results can be implemented in an efficient manner.

This paper is organized as follows. In Section 2 we define a dynamical network and the notion of intrinsic stability. In Section 3 we introduce networks with constant time-delays and show that these delayed systems have the same fixed points as their undelayed versions. In Section 4 we define switched networks, i.e. switched systems, and give our Main Results 1 and 2, which show that intrinsically stable switched systems have the same asymptotic state irrespective of initial condition if intrinsically stable and intrinsically stable networks with time-varying time-delays have a globally attracting fixed point, respectively. We then apply this theory to linear systems with distinct delayed and undelayed interactions, which allow us to compare our results to some of the most well-studied time-delayed systems. In Section 5 we extend our results to a more general class of switched networks and similarly compare our results to a number of well-studied switching systems. In Section 6 we introduce some analytical and computational considerations related to determining whether a network is intrinsically stable. Section 7 contains some remarks about future work and applications of this theory. The Appendix contains the proofs of our results.

2 Dynamical Networks

A network is composed of a set of elements, which are the individual units that make up the network and a collection of interactions between these elements. An interaction between two network elements can be thought of as an element’s ability to influence the behavior of the other network element. More generally, there is a directed interaction from the $j^{th}$ to the $i^{th}$ elements of a network if the $j^{th}$ network element can influence the state of the $i^{th}$ network element (where there may be no influence of the $i^{th}$ network element on the $j^{th}$). The dynamics of a network can be formalized as follows:

Definition 1**.**

(Dynamical Network)* Let $(X_{i},d_{i})$ be a complete metric space for $1\leq i\leq n$. Let $(X,d_{max})$ be the complete metric space formed by endowing the product space $X=\bigoplus_{i=1}^{n}X_{i}$ with the metric*

[TABLE]

Let $F\mathrel{\mathop{\mathchar 58\relax}}X\to X$ be a continuous map, with $i^{th}$ component function $F_{i}\mathrel{\mathop{\mathchar 58\relax}}X\to X_{i}$ given by

[TABLE]

where it is understood that there may be no actual dependance of $F_{i}$ on $x_{j}$. The dynamical system $(F,X)$ generated by iterating the function $F$ on $X$ is called a dynamical network. If an initial condition $\mathbf{x}^{0}\in X$ is given, we define the $k^{th}$ iterate of $\mathbf{x}^{0}$ as $\mathbf{x}^{k}=F^{k}(\mathbf{x}^{0})$, with orbit $\{F^{k}(\mathbf{x}^{0})\}_{k=0}^{\infty}=\{\mathbf{x}^{0},\mathbf{x}^{1},\mathbf{x}^{2},\ldots\}$ in which $\mathbf{x}^{k}$ is the state of the network at time $k\geq 0$.

The component function $F_{i}=F_{i}(x_{1},x_{2},\ldots,x_{n})$ describes the dynamics and interactions with the $i^{th}$ element of the network, where there is a directed interaction between the $i^{th}$ and $j^{th}$ elements if $F_{i}$ actually depends on $x_{j}$. The function $F\mathrel{\mathop{\mathchar 58\relax}}X\rightarrow X$ describes all interactions of the network $(F,X)$. For the initial condition $\mathbf{x}^{0}\in X$ the state of the $i^{th}$ element at time $k\geq 0$ is $x^{k}_{i}=(F^{k}(\mathbf{x}^{0}))_{i}\in X_{i}$ so that $X_{i}$ is the element’s state space. The state space $X=\bigoplus_{i=1}^{n}X_{i}$ is the collective state space of all network elements.

While the definition of a dynamical network allows for the function $F$ to be defined on general products of complete metric spaces, for the sake of intuition and direct applications of the theory we develop in this paper, our examples will focus on dynamical networks in which $X=\mathbb{R}^{n}$ with the infinity norm $\left\|\mathbf{x}\right\|_{\infty}=\max_{i}\left\lvert x_{i}\right\rvert$. To give a concrete example of a dynamical network and to illustrate results throughout this paper, we will use Cohen-Grossberg neural (CGN) networks.

Example 2.1**.**

(Cohen-Grossberg Neural Networks) For $W\in\mathbb{R}^{n\times n}$, $\sigma\mathrel{\mathop{\mathchar 58\relax}}\mathbb{R}\to\mathbb{R}$, and $c_{i},\epsilon\in\mathbb{R}$ for $1\leq i\leq n$ let $(C,\mathbb{R}^{n})$ be the dynamical network with components

[TABLE]

which is a special case of a Cohen-Grossberg neural network in discrete-time [21]. The function $\sigma$ is assumed to be bounded, differentiable, and monotonically increasing, with Lipschitz constant $K$, that is,

[TABLE]

for all $x,y\in\mathbb{R}$.

In a Cohen-Grossberg neural network the variable $x_{i}$ represents the activation of the $i^{th}$ neuron. The function $\sigma$ is a bounded monotonically increasing function, which describes the $i^{th}$ neuron’s response to inputs. The matrix $W$ gives the interaction strengths between each pair of neurons and describes how the neurons are connected within the network. The constants $c_{i}$ indicate constant inputs from outside the network.

In a globally stable dynamical network $(F,X)$, the state of the network tends toward an equilibrium irrespective of its initial condition. That is, the network has a globally attracting fixed point $\mathbf{y}\in X$ such that for any $\mathbf{x}\in X$, $F^{k}(\mathbf{x})\to\mathbf{y}$ as $k\to\infty$.

Global stability is observed in a number of important systems including neural networks [21, 22, 19, 23, 20], epidemic models [24], and the study of congestion in computer networks [25]. In such systems the globally attracting equilibrium is typically a state in which the network can carry out a specific task. Whether or not this equilibrium stays stable depends on a number of factors including external influences but also internal processes such as the network’s own growth, both of which can destabilize the network.

To give a sufficient condition under which a network $(F,X)$ is stable, we define a Lipschitz matrix (called a stability matrix in [26]).

Definition 2**.**

(Lipschitz Matrix)* For $F\mathrel{\mathop{\mathchar 58\relax}}X\to X$ suppose there are finite constants $a_{ij}\geq 0$ such that*

[TABLE]

Then we call $A=[a_{ij}]\in\mathbb{R}^{n\times n}$ a Lipschitz matrix of the dynamical network $(F,X)$.

It is worth noting that if $A$ is a Lipschitz matrix of a dynamical network then any matrix $B\preceq A$, where $\preceq$ denotes the element-wise inequality, is also a Lipschitz matrix of the network. However, if the function $F\mathrel{\mathop{\mathchar 58\relax}}X\to X$ is piecewise differentiable and each $X_{i}\subseteq\mathbb{R}$ then the matrix $A\in\mathbb{R}^{nxn}$ given by

[TABLE]

is the Lipschitz matrix of minimal spectral radius of $(F,X)$ (see [26]). From a computational point of view, the Lipschitz matrix $A=[a_{ij}]$ of $(F,X)$ can be more straightforward to find by use of Equation (2) if the function $F\mathrel{\mathop{\mathchar 58\relax}}X\to X$ is differentiable, compared to the more general formulation in Definition 2.

Using Equation (2) it follows that the Lipschitz matrix $A$ of the Cohen-Grossberg neural network from Example 2.1 is given by

[TABLE]

It is straightforward to verify that a Lipschitz matrix exists for a dynamical network $(F,X)$ if and only if the mapping $F$ is Lipschitz continuous. The idea is to use the Lipschitz matrix to simplify the stability analysis of nonlinear networks, using the following theorem of [26]. Here

[TABLE]

denotes the spectral radius of a matrix $A$, where $\sigma(A)$ are the eigenvalues of $A$.

Theorem 1**.**

(Network Stability)* Let $A$ be a Lipschitz matrix of a dynamical network $(F,X)$. If $\rho(A)<1$, then $(F,X)$ is stable.*

It is worth noting that if we use the Lipschitz matrix $A$ to define the dynamical network $(G,X)$ by $G(\mathbf{x})=A\mathbf{x}$ then $(G,X)$ is stable if and only if $\rho(A)<1$. Thus, a Lipschitz matrix of a dynamical network $(F,X)$ can be thought of as the worst-case linear approximation to $F$. If this approximation has a globally attracting fixed point, then the original dynamical network $(F,X)$ must also be stable. Note, however, that the condition that $\rho(A)<1$ is sufficient but not necessary for $(F,X)$ to be stable. In fact, this stronger condition implies much more than network stability, so following the convention introduced in [26] we assign it the name of intrinsic stability.

Definition 3**.**

(Intrinsic Stability)* Let $A\in\mathbb{R}^{nxn}$ be a Lipschitz matrix of a dynamical network $(F,X)$. If $\rho(A)<1$, then we say $(F,X)$ is intrinsically stable.*

The Cohen-Grossberg neural network $(C,\mathbb{R}^{n\times n})$ has the stability matrix $A=\left\lvert 1-\epsilon\right\rvert I+K\left\lvert W\right\rvert$ given by Equation (3) with spectral radius

[TABLE]

Here, $\left\lvert W\right\rvert\in\mathbb{R}^{n\times n}$ is the matrix $W$ in which we take the absolute value of each entry. Thus, $(C,\mathbb{R}^{n\times n})$ is intrinsically stable if $\left\lvert 1-\epsilon\right\rvert+K\rho(\left\lvert W\right\rvert)<1$.

3 Constant-Time-Delayed Dynamical Networks

As mentioned in the introduction, the dynamics of most real networks are time-delayed. That is, an interaction between two network elements will typically not happen instantaneously but will be delayed due to either the physical separation of these elements, their finite processing speeds, or be delayed due to other factors. We formalize this phenomenon by introducing a delay distribution matrix $D=[d_{ij}]$ into a dynamical network $(F,X)$, where each $d_{ij}$ is a nonnegative integer denoting the constant number of discrete time-steps by which the interaction from the $j^{th}$ network element to the $i^{th}$ network element is delayed.

Definition 4**.**

(Constant Time-Delayed Dynamical Network)**

Let $(F,X)$ be a dynamical network and $D=[d_{ij}]\in\mathbb{N}^{n\times n}$ a delay distribution matrix with $\max_{i,j}d_{ij}\leq L$, a bound on the delay length. Let $X_{L}$, the extension of $X$ to delay-space, be defined as

[TABLE]

Componentwise, define $F_{D}\mathrel{\mathop{\mathchar 58\relax}}X_{L}\to X_{L}$ by

[TABLE]

and

[TABLE]

where $F_{i}\mathrel{\mathop{\mathchar 58\relax}}X\to X_{i}$ is the $i^{th}$ component function of $F$ for $i=1,2,\ldots,n$. Then $(F_{D},X_{L})$ is the delayed version of $F$ corresponding to the fixed-delay distribution $D$ with delay bound $L$.

We order the component spaces of $X_{L}$ in the following way. If $\mathbf{x}\in X_{L}$ then

[TABLE]

where $x_{i,\ell}\in X_{i,\ell}$ for $i=1,2,\ldots,n$ and $\ell=0,1,\ldots,L$.

The formalization in Definition 4 captures the idea of adding a time-delay of length $d_{ij}\leq L$ to an interaction: Each $X_{i}$ is effectively copied $L$ times, and past states of the $i^{th}$ element are passed down this chain by the identity component functions $(F_{D})_{i,\ell+1}$ for $0\leq\ell\leq L-1$ in Equation (4). When a state of the $i^{th}$ element has been passed through the chain $d_{ij}$ times over $d_{ij}$ time-steps it then influences the $i^{th}$ network element, as described by the entry-wise substitutions of $x_{j,d_{ij}}$ for $x_{j}$ in $(F_{D})_{i,0}$ in Equation (5).

Example 3.2**.**

Consider a simple 2-neuron version of a Cohen-Grossberg neural network $(C,X)$ given by

[TABLE]

in which $X=\mathbb{R}^{2}$. For the delay distribution $D$ given by

[TABLE]

which has a maximum delay of $L=3$ the time-delayed network $(C_{D},X_{3})$ is given by

[TABLE]

in which $X_{3}=\mathbb{R}^{8}$. The time-delayed network $(C_{D},X_{3})$ is the same as the original network $(C,X)$ except that the state of $x_{1,0}$ gets passed through one identity mapping before it is input into $F_{1}$ and twice before it is input into $F_{2}$. Similarly, $x_{2,0}$ gets passed through one identity mapping before it is input into $F_{1}$ and three identity mappings before it is input into $F_{2}$.

A natural question is to ask is whether constant time-delays affect the stability of a network. We note that if

[TABLE]

$c_{1}=c_{2}=0$, $\sigma(x)=tanh(x)$, and $\epsilon=\frac{2}{5}$ then the dynamical network $(C,X)$ given in Equation (6) is stable as can be seen in Figure (1) (left). However, the time-delayed version $(C_{D},X_{3})$ of this network is unstable as is shown in the same figure (right). That is, the time-delays given by the delay distribution $D$ have a destabilizing effect on this network.

An important fact about the network constructed in this example is that its spectral radius

[TABLE]

That is, although $(C,X)$ is stable it is not intrinsically stable.

In [26], the authors demonstrate that intrinsically stable systems are resilient to the addition of constant time-delays, as is stated in the following theorem.

Theorem 2**.**

(Intrinsic Stability and Constant Delays)* Let $(F,X)$ be a dynamical network and $D=[d_{ij}]$ a delay-distribution matrix. Let $L$ satisfy $\max_{i,j}d_{ij}\leq L$. Then $(F,X)$ is intrinsically stable if and only if $(F_{D},X_{L})$ is intrinsically stable.*

Beyond maintaining stability, we note that any fixed point(s) of an undelayed network $(F,X)$ will also be fixed point(s) of any delayed version $(F_{D},X_{L})$. This is formalized in the following proposition, and proven in the Appendix. Before stating this proposition, we require the following definition.

Definition 5**.**

(Extension of a Point to Delay-Space)* Let $E_{L}(\mathbf{x})\in X_{L}$ be equal to $L+1$ copies of $\mathbf{x}\in X$ stacked into a single vector, namely*

[TABLE]

Proposition 1**.**

(Fixed Points of Delayed Networks)* Let $\mathbf{x}^{*}$ be a fixed point of a dynamical network $(F,X)$. Then for all delay distributions $D$ with $\max_{ij}d_{ij}\leq L$, $E_{L}(\mathbf{x}^{*})$ is a fixed point of $(F_{D},X_{L})$.*

As an immediate consequence of Proposition 1 and Theorem 2, if an undelayed network $(F,X)$ is intrinsically stable with a globally attracting fixed point $\mathbf{x}^{*}\in X$, then the delayed version $(F_{D},X_{L})$ will have the “same” globally attracting fixed point $E_{L}(\mathbf{x}^{*})$, in that $\mathbf{x}^{*}$ is the restriction of $\mathbf{y}=E_{L}(\mathbf{x}^{*})$ to the first $n$ component spaces of $X_{L}$. Thus, the asymptotic dynamics of an intrinsically stable network and any version of the network with constant time delays are essentially identical.

Example 3.3**.**

Consider again the Cohen-Grossberg neural network $(C,X)$ and delay matrix $D$ given in Example 3.2 where $W$ and $\sigma$ are as before but $\epsilon=\frac{4}{5}$, $c_{1}=-1$ and $c_{2}=1$. Since $\left\lvert 1-\epsilon\right\rvert+\rho(\left\lvert W\right\rvert)=.95<1$ then $(C,X)$ is intrinsically stable with the globally attracting fixed point $\mathbf{x}^{*}=(-.386,1.595)$ as seen in Figure 2 (left). Since $(C,X)$ is intrinsically stable then Theorem 2 together with Proposition 1 imply that not only is $(C_{D},X_{3})$ stable but its globally attracting fixed point is $E_{3}(\mathbf{x}^{*})$. This is shown in Figure 2 (right).

These results justify the modeling choice of ignoring constant time-delays when analyzing intrinsically stable real-world networks. However, these results do not account for the potential of time-dependent delays arising from external or stochastic influences, etc. The main results of this paper, presented in the next section, focus on strengthening the conclusion of Theorem 2.

4 Time-Varying Time-Delayed Dynamical Networks

As the title of this section suggests, constant time-delays are not the only type of time delays that can occur in dynamical networks. More importantly, time-delays that vary with time occur in many real-world networks and in such systems are a significant source of instability [5, 6, 7]. It is worth noting that modeling such delays introduces even more complexity into models of dynamical networks that can already be quite complicated. This can hinder the tractability of analyzing such systems.

In order to define a network with time-varying time-delays, we first define the more general concept of a switched network.

Definition 6**.**

(Switched Network)* Let $M$ be a set of Lipschitz continuous mappings on $X$, such that for every $F\in M$, $(F,X)$ is a dynamical network. Then we call $(M,X)$ a switched network on $X$. Given some sequence $\{F^{(k)}\}_{k=1}^{\infty}\subset M$, we say that $(\{F^{(k)}\}_{k=1}^{\infty},X)$ is an instance of $(M,X)$, with orbits determined at time $k$ by the function*

[TABLE]

For the switched network $(M,X)$ we construct a Lipschitz set $S$ consisting of a set of $n\times n$ matrices as follows: For each $F\in M$, contribute exactly one Lipschitz matrix $A$ of $(F,X)$ to the set $S$.

$(M,X)$ is an ensemble of dynamical systems formed by taking all possible sequences of mappings $\{F^{(k)}\}_{k=1}^{\infty}\subset M$. For a switched network $(M,X)$, the set $S$ serves an analogous purpose to the Lipschitz matrix $A$ of a dynamical network $(F,X)$, as we soon demonstrate. Before describing this we consider the following example.

Example 4.4**.**

Let $(P,\mathbb{R}^{2})$ and $(Q,\mathbb{R}^{2})$ be the simple dynamical networks given by

[TABLE]

For $M=\{P,Q\}$ let $\{F^{(k)}\}_{k=1}^{\infty}$ be the sequence that alternates between $P$ and $Q$, i.e. $F^{(k)}=P$ if $k$ is odd and $F^{(k)}=Q$ if $k$ is even. Note that if we let $U=Q\circ P$ then

[TABLE]

Since $\mathcal{F}^{(2k)}(\mathbf{x})=U\circ\ldots\circ U(\mathbf{x})$ for $\mathbf{x}\in\mathbb{R}^{2}$ then, as $U(\mathbf{x})$ is a linear system $\lim_{k\to\infty}\mathcal{F}^{(2k)}(\mathbf{x})=\infty$ for any $\mathbf{x}\neq 0$. This is despite the fact that both $(P,\mathbb{R}^{2})$ and $(Q,\mathbb{R}^{2})$ are intrinsically stable both having the globally attracting fixed point $\mathbf{0}$.

The issue is that although the spectral radius of both $P$ and $Q$ in this example are arbitrarily small, their joint spectral radius is not.

Definition 7**.**

(Joint Spectral Radius)* Given some $\mathbf{z}^{0}\in\mathbb{R}^{n}$ and some set of matrices $S\subset\mathbb{R}^{n\times n}$, let $\mathbf{z}^{k}=A_{k}\ldots A_{2}A_{1}\mathbf{z}^{0}$ for some sequence $\{A_{i}\}_{i=1}^{\infty}\subset S$. The joint spectral radius $\overline{\rho}(S)$ of the set of matrices $S$ is the smallest value $\overline{\rho}\geq 0$ such that for every $\mathbf{z}^{0}\in\mathbb{R}^{n}$ there is some constant $C>0$ for which*

[TABLE]

It is known that $\{\mathbf{z}^{k}\}_{k=1}^{\infty}$ converges to the origin for all $\mathbf{z}^{0}\in\mathbb{R}^{n}$ if and only if $\overline{\rho}(S)<1$ [27]. This allows us to state the following result regarding the asymptotic behavior of a nonlinear switched network $(M,X)$ whose Lipschitz set $S$ satisfies $\overline{\rho}(S)<1$.

Main Result 1**.**

(Independence of Initial Conditions for Switched Networks)* Let $S$ be a Lipschitz set of a switched network $(M,X)$ satisfying $\overline{\rho}(S)<1$, and let $(\{F^{(k)}\}_{k=1}^{\infty},X)$ be an instance of $(M,X)$. Then for all initial conditions $\mathbf{x}^{0},\mathbf{y}^{0}\in X$, there exists some $C>0$ such that*

[TABLE]

Additionally, if $\mathbf{x}^{*}$ is a shared fixed point of $(F,X)$ for all $F\in M$, then $\lim_{k\to\infty}\mathscr{F}^{k}(\mathbf{x}^{0})=\mathbf{x}^{*}$ for all initial conditions $\mathbf{x}^{0}\in X$.

Hence, if the joint spectral radius of the Lipschitz set $S$ of $M$ is less than $1$, all orbits in a switched network become asymptotically close to one another as time goes to infinity. Even if this limit-orbit is not convergent to any fixed point, this result implies an asymptotic independence to initial conditions. An example of this is the following.

Example 4.5**.**

Let $(G,X)$ and $(H,X)$ be the Cohen-Grossberg neural networks given by

[TABLE]

respectively, in which $\sigma(x)=\tanh(x)$, $\epsilon_{1}=\frac{4}{5}$, $\epsilon_{2}=\frac{3}{10}$, $c_{1}=d_{2}=-1$, and $c_{2}=d_{1}=1$. Setting $M=\{G,H\}$ then $S$ is the Lipschitz set of $M$ given by

[TABLE]

It can be shown that the joint spectral radius $\overline{\rho}(S)=.95$ (by use of Proposition 2 in Section 5). As this is less than 1, then for any instance $(\{F^{(k)}\}_{k=1}^{\infty},X)$ of $M$ and any initial conditions $\mathbf{x}^{0}$ and $\mathbf{y}^{0}$, we have

[TABLE]

by the Main Result 1. This can be seen in Figure 3 (left) where $(\{F^{(k)}\}_{k=1}^{\infty},X)$ is the instance given by $\{F^{(k})\}_{k=1}^{\infty}=\{G,G,G,H,H,H,\dots\}$.

If we set $c_{1}=c_{2}=d_{1}=d_{2}=0$ in both $(G,X)$ and $(H,X)$ then both systems have the shared fixed point $\mathbf{0}$. In this case Main Result 1 indicated that any instance $(\{F^{(k)}\}_{k=1}^{\infty},X)$ of $M$ will be stable with the globally attracting fixed point $\mathbf{0}$. This is shown in Figure 3 (right) where again $\{F^{(k})\}_{k=1}^{\infty}=\{G,G,G,H,H,H,\dots\}$.

Thus, as might be expected from the complicated nature of a switched system, the condition $\overline{\rho}(S)<1$ alone is not able to match the strong implication of global stability, as $\rho(A)<1$ does for a dynamical network (see Theorem [2]). Furthermore, it is often notoriously difficult to compute or approximate the joint spectral radius $\overline{\rho}(S)$ of a general set of matrices $S$ [27, 33].

Somewhat surprisingly, these issues are resolved when our switched system arises from a network experiencing time-varying time-delays. In this case, the computation of the joint spectral radius reduces to computing the spectral radius of the Lipschitz matrix of the original undelayed dynamical network $(F,X)$, which for even large systems can be done efficiently using the power method [11]. This provides a general and computationally efficient method for verifying asymptotic stability despite time-varying time-delays.

Main Result 2**.**

(Intrinsic Stability and Time-Varying Time-Delayed Networks)* Suppose $(F,X)$ is intrinsically stable with $\rho(A)<1$, where $A\in\mathbb{R}^{n\times n}$ is a Lipschitz matrix of $F$ and $\mathbf{x}^{\ast}$ is the network’s globally attracting fixed point. Let $L>0$ and*

[TABLE]

and let $S_{d}$ be the Lipschitz set of $M_{d}$.

Then $E_{L}(\mathbf{x}^{*})$ is a globally attracting fixed point of every instance $(\{F_{D^{(k)}}\}_{k=1}^{\infty},X_{L})$ of $(M_{d},X_{L})$. Furthermore, $\overline{\rho}(S_{d})=\rho(A_{L})<1$, where

[TABLE]

Hence, any intrinsically stable dynamical network $(F,X)$ retains convergence to the same equilibrium even when it experiences time-varying time-delays. This extends Theorem 2.3 of [26] to the much larger and more complicated class of switching-delay networks. Furthermore, note that intrinsic stability is a delay-independent result, which makes no assumption regarding the rate of growth of the time delays. In Main Result 2 we call $\overline{\rho}(S_{d})$ the convergence rate of the system, since it provides the exponential bound on the rate at which all orbits converge to the fixed point $E_{L}(\mathbf{x}^{*})$.

It is worth emphasizing that as a consequence of this result, to determine the asymptotic behavior of a switched network $(M,X_{L})$ with $M=\{F_{D}|D\in\mathbb{N}^{n\times n}\text{ with }\max_{ij}d_{ij}\leq L\}$, in which the presence and magnitude of time delays is not exactly known, it suffices to study the dynamics of the much simpler undelayed system $(F,X)$.

Example 4.6**.**

(Periodic and Stochastic Time-Varying Delays)* Consider the intrinsically stable Cohen-Grossberg neural network $(C,X)$ from Example 3.3 with the exception that we let $\{D^{(k)}\}_{k=1}^{\infty}$ be the sequence of delay distributions given by*

[TABLE]

These time-varying delays are periodic with period $30$ and $L=5$. The result of these delays on the network’s dynamics can be seen in Figure 4 (left) where we let $(C_{P},X_{5})$ denote the network $(C,X)$ with time-varying time-delays given by (8). Note that although the trajectories are altered by these delays they still converge to the same fixed point $\mathbf{x}^{*}=(-.386,1.595)$ as in the undelayed and constant time-delayed networks (cf. Figure 2) as guaranteed by Main Result 2.

If instead we let

[TABLE]

be the random matrix in which each entry is sampled uniformly from the integers $\{0,1,\ldots,10\}$ the resulting switched network is still stable, as guaranteed by Main Result 2. The network’s trajectories still converge to the point $\mathbf{x}^{*}=(-.386,1.595)$ as shown in Figure 4 (right) where we let $(C_{U},X_{10})$ denote the network $(C,X)$ with time-varying time-delays given by 9.

4.1 Application: Linear Systems with Distinct Delayed and Undelayed Interactions

A number of papers have published delay-dependant results regarding time-varying time-delayed systems whose delayed interactions are separated from the undelayed interactions (see , for instance, [14, 15, 16, 17, 18]). By delay-dependent we mean that the criteria that determines whether a network is stable depends on the length of the delays the network experiences, as opposed to the delay-independent results of Section 4. We demonstrate how to analyze these by use of Main Result 2, and compare our results to those in the literature.

Consider a linear system of the form

[TABLE]

where $A,B\in\mathbb{R}^{n\times n}$ are constant matrices, and $\tau(k)$ is a positive integer representing the magnitude of the time-varying time-delay, bounded by some $L>0$. Here $A$ and $B$ represent distinct weights of the delayed and undelayed interactions.

The minimally delayed version of system (10) is given by

[TABLE]

which we can express in terms of a single transition matrix $\widetilde{A}$ as

[TABLE]

As in [28], we say that system (12) is the lifted representation of system (11).

Now observe that we can represent system (10) in the notation of Main Result 2 as the system $(F,\mathbb{R}^{2n})$ where

[TABLE]

where, since $F$ is linear, the Lipschitz matrix of $F$ is given by $\left|\widetilde{A}\right|\in\mathbb{R}^{2n\times 2n}$ and the zero vector $\mathbf{0}\in\mathbb{R}^{n}$ is a fixed point. Then system (10) is the switched system instance $(\{F_{D^{(k)}}\}_{k=1}^{\infty},X_{L})$ obtained from the sequence of delay distributions

[TABLE]

ensuring that delays only occur to the delayed interactions modeled by $B$, with magnitude $\tau(k)\leq L$.

It follows immediately from Main Result 2 that the system (10) is stable for arbitrary large delay bounds $L$ when $(F,\mathbb{R}^{2n})$ is intrinsically stable, that is when $\rho(\widetilde{A})<1$ is satisfied.

Example 4.7**.**

(Intrinsically Stable)* Consider system (10) with*

[TABLE]

The transition matrix of the lifted representation is

[TABLE]

which satisfies $\rho(\left|\widetilde{A}\right|)\approx 0.822<1$. Then this system is intrinsically stable, so is in fact stable for an arbitrarily large delay bound $L>0$. In the following table, we compare this delay-independent result with the delay-dependent results of [14, 15, 16, 17, 18]:

[TABLE]

It is worth noting that the methods of [14, 15, 16, 17, 18] employ techniques involving linear matrix inequalities and Lyapunov functionals. These methods are avoided with a straight-forward application of Main Result 2, which only requires computing the spectral radius of single $4\times 4$ matrix. As elaborated in Section 6, checking intrinsic stability is extremely computationally efficient. Furthermore, because intrinsic stability is a delay-independent result, it becomes immediately clear that an intrinsically stable network is stable for arbitrarily large delay bounds $L$.

Beyond improving the results described in Example 4.7, in a similar manner, we can also improve the results of Example 2 of [29]. In doing so we replicate the delay-independent result of Example 6.1 of [30], in which the system is found to be stable for arbitrarily large $L$. However, we do so in a much more computationally efficient manner, without having to solve a series of linear matrix inequalities (see [29]).

There are systems of the form (10) which are not intrinsically stable, in which case delay-dependent results provide greater insight.

Example 4.8**.**

(Not Intrinsically Stable)* Consider system (10) with*

[TABLE]

The transition matrix of the lifted representation is

[TABLE]

which satisfies $\rho(\left|\widetilde{A}\right|)=1$. Hence this system is not intrinsically stable, even though [14] showed that it is stable for all $0\leq L\leq 9.61\times 10^{8}$.

Even though some systems that are not intrinsically stable turn out to be stable, at least for certain types of delays, the fact that intrinsic stability can be verified with relatively little effort may be reason enough to check. In fact, it may be the case that a spectral radius slightly above or equal to 1 is an indication that a system’s stability is resilient to time delays as in Example 4.8. However, this is still an open question. For further analysis of the computational complexity of checking intrinsic stability, see section 6.

5 Row-Independence Closure of Switched Networks

Using Main Result 1 and Main Result 2, we can extend our analysis of systems with time-varying time-delays to a more general class of switched networks.

Definition 8**.**

(Row-Independence Closure)* Let $(M,X)$ be a switched network with Lipschitz set $S$. Then we denote $RI(S)$, the row-independence closure of $S$, by*

[TABLE]

where $\mathbf{a}^{(i)}_{i}$ denotes the $i^{th}$ row of the $i^{th}$ matrix $A^{(i)}$.

The idea is that row-independence indicates that there is no conditional relationship between rows of the matrices in $RI(S)$. Note that $S\subset RI(S)$.

The row-independence closure provides a computationally efficient sufficient condition for satisfying the hypothesis of Main Result 1. The following proposition follows directly from Definition 8 and the results of [27] (restated as Theorem 5 in the Appendix).

Proposition 2**.**

(Row-Independence Closure, Intrinsic Stability, and Switched Networks)* Let $(M,X)$ be a switched network with Lipschitz set $S$. Then*

[TABLE]

This allows for the following extension of Main Result 2, which provides a sufficient condition ensuring that when time-varying time-delays are applied to a stable already-switched system, the new resulting switched system retains stability.

Main Result 3**.**

(Intrinsic Stability and Row-Independent Switched Networks)* Let $(M,X)$ be a switched network with Lipschitz set $S$. Assume $\mathbf{x}^{*}$ is a shared fixed point of $(F,X)$ for all $F\in M$ and $\rho(A)<1$ for all $A\in RI(S)$. Let $L>0$ and*

[TABLE]

and let $S_{d}$ be the Lipschitz set of $M_{d}$.

Then $E_{L}(\mathbf{x}^{*})$ is a globally attracting fixed point of every instance $(\{F^{(k)}_{D^{(k)}}\}_{k=1}^{\infty},X_{L})$ of $(M_{d},X_{L})$. Furthermore,

[TABLE]

where given some $A\in\mathbb{R}^{n\times n}$, $A_{L}$ is defined as

[TABLE]

Thus, a switched network with a shared fixed point which also satisfies $\rho(A)<1$ for all $A\in RI(S)$ retains convergence to the same equilibrium, even when it experiences time-varying time-delays. This extends Main Result 2 further to the even more complicated class of switched networks with time-varying time-delays.

5.1 Application: Switched Linear Systems with Distinct Delayed and Undelayed Interactions

We now extend our analysis of Section 4.1 to the case where system (10) is also a switched system:

[TABLE]

where as before, each $A_{\sigma(k)},B_{\sigma(k)}\in\mathbb{R}^{n\times n}$ are constant matrices indexed by $\sigma(k)$, and $\tau(k)$ is a positive integer representing the magnitude of the time-varying time-delay, bounded by some $L>0$. The difference between system (13) and system (10) is that in system (13) there are multiple possibilities for the transition weights of the delayed and undelayed interactions given by $A_{\sigma(k)}$ and $B_{\sigma(k)}$, respectively.

The minimally delayed version of system (13) is given by

[TABLE]

so the lifted version of system (14) is

[TABLE]

Now observe that we may represent system (13) in the notation of Main Result 3 as the system $(M_{0},\mathbb{R}^{2n})$ where

[TABLE]

where, since each $F$ is linear, the Lipschitz matrix of $F$ is given by $\left|\widetilde{A}_{\sigma(k)}\right|$ and the zero vector $\mathbf{0}\in\mathbb{R}^{n}$ is a shared fixed point. Then system (13) is the switched system instance $(\{F^{(k)}_{D^{(k)}}\}_{k=1}^{\infty},X_{L})$ obtained from the sequence of transition matrices $\widetilde{A}_{\sigma(k)}$ and the sequence of delay distributions

[TABLE]

once the sequences $\sigma(k)$ and $\tau(k)$ are determined. This ensures that delays only occur to the delayed interactions modeled by $B_{\sigma(k)}$, with magnitude $\tau(k)$. It follows immediately from Main Result 3 that the system (13) is stable for arbitrary large delay bounds $L$ when $(M_{0},\mathbb{R}^{2n})$ satisfies the relatively simple condition $\rho(A)<1$ for all $A\in RI(S)$ of the Lipschitz set $S$ of $M_{0}$.

We apply our analysis to two examples from [31] to demonstrate the effectiveness of Main Result 3.

Example 5.9**.**

(Row-Independent Switched Network)* Consider system (13) with*

[TABLE]

[TABLE]

In Example 1 of [31], this system was shown to be exponentially stable for a delay bound of $L=13$.

The set $M_{0}$ of lifted transition matrices consists of the following four matrices:

[TABLE]

[TABLE]

Note that here $M_{0}=RI(M_{0})$. Furthermore, $\rho(\left|\widetilde{A}_{1}\right|)=\rho(\left|\widetilde{A}_{3}\right|)\approx 0.59<1$ and $\rho(\left|\widetilde{A}_{2}\right|)=\rho(\left|\widetilde{A}_{4}\right|)\approx 0.39<1$. Hence, this switched system is intrinsically stable, so is in fact stable for any bound $L<\infty$, no matter the sequence of $A_{\sigma(k)}$ and $B_{\sigma(k)}$ that are chosen. It is worth mentioning that this conclusion was reached without having to iteratively solve a system of linear matrix inequalities as in [31].

In Example 5.9, the set $M_{0}$ trivially satisfied $M_{0}=RI(M_{0})$. In our next example, we consider the more nuanced case where $M_{0}$ is a proper subset of $RI(M_{0})$.

Example 5.10**.**

(Row-Independent Closure of a Switched Network)* Consider the following modified version of system (13), where we hold $A$ and $B$ constant in time, but add in a switching control term $\mathbf{c}_{\sigma(k)}\mathbf{u}^{k}$:*

[TABLE]

This system has the minimally-delayed lifted representation

[TABLE]

In Example 2 of [31], system 16 was shown to be stable with a upper delay bound $L=2$ for

[TABLE]

and

[TABLE]

In this case we have three matrices in $M_{0}$:

[TABLE]

[TABLE]

However, $RI(M_{0})$ contains $\widetilde{A}_{1},\widetilde{A}_{2},\widetilde{A}_{3}$ as well as the following fourth matrix

[TABLE]

Here $\rho(\left|\widetilde{A}_{1}\right|)\approx 0.9097<1$, $\rho(\left|\widetilde{A}_{2}\right|)\approx 0.9106<1$, $\rho(\left|\widetilde{A}_{1}\right|)\approx 0.9104<1$, $\rho(\left|\widetilde{A}_{1}\right|)\approx 0.9099<1$. Thus this switched network is intrinsically stable. Therefore, using Main Result 3 we have been able to efficiently show that this system is stable for arbitrarily large delay bounds $L$.

It is worth emphasizing that while each of the examples in this section considers linear switched networks, the analysis applies directly to nonlinear switched networks as well, once the stability set $M_{0}$ of the various nonlinear mappings is known.

6 Analytical and Computational Considerations

We now summarize how these results can be used to analyze dynamical systems with a network structure. That is, for a dynamical network with either constant time-delays $(F_{D},X_{L})$ or network with time-varying time-delays with instances given by $(\{F_{D^{(k)}}\},X_{L})$ using the following steps:

Step 1: Consider the simpler undelayed version of the network $(F,X)$ where all interactions occur instantaneously. For a system given by Equation (10) consider the minimally delayed version of the system given by Equation (11).

Step 2: Find a Lipschitz matrix $A$ of the network by Equation (2) if the mapping $F$ is piecewise differentiable and $X=\mathbb{R}^{n}$, otherwise directly determine the Lipschitz constants $A=[a_{ij}]$ by use of Definition 2. Recall that there are infinitely many Lipschitz matrices $A$ for a given dynamical network. Ideally we would like to find one which minimizes the spectral radius $\rho(A)$, as this improves our estimate of the convergence rate of the system to its unique equilibrium, if $\rho(A)<1$. However, in certain cases it may simplify analysis considerably to merely find a bound for each entry of the Lipschitz matrix. Specifically, a bound that shows $\rho(A)<1$ for some Lipshitz matrix $A$.

Step 3: The spectral radius $\rho(A)$ can be computed efficiently in $\mathcal{O}(mn)$ time by use of the power method, where $m$ is the number of nonzero entries in $A$ [11]. As soon as a single Lipschitz matrix $A$ of $(F,X)$ satisfies $\rho(A)<1$, even if this $A$ does not have minimal spectral radius, the results of Theorem 2 and Main Result 2 apply. These results guarantee that all delayed versions of $(F,X)$ are stable, even if the delays are varying in time so long as the magnitude of these delays are eventually bounded by some $L<\infty$. Moreover, the convergence rate of the delayed system is given by $\rho(A_{L})$, where $A_{L}$ is defined in Main Result 2. Since $A_{L}$ is sparse with $m+nL$ nonzero entries, $\rho(A_{L})$ can also be computed in $\mathcal{O}(mnL+n^{2}L^{2})$ time using the power method.

7 Conclusion

The method described in this paper for determining whether a network is stable or can be destabilized by time-delays has a number of advantages over other methods. First, one need not consider the system itself but rather the simpler undelayed, and therefore lower-dimensional, version of the system. Second, the method(s) described here, at least for networks, require only the computation and spectral analysis of a single matrix rather than the use of Lyapunov-type methods, Linear Matrix Inequalities, or Semi-Definite Programming methods. Hence, very large systems can be analyzed using this method under the condition that their Lipschitz matrix can be efficiently computed. Moreover, if a system (process) can be shown (designed) to be intrinsically stable, there is no need to formally include delays in the system (model). The reason is that delays do change the qualitative dynamics of the system and it will same asymptotic state whether or not its delays are included. As has been shown, this is not the case for general dynamical networks.

One question that remains open is if a system is not intrinsically stable does there exist a time-delayed version of the system that is unstable. Another is that certain systems may be only locally intrinsically stable, meaning that delaying certain network interactions may have no effect on the network’s stability while delaying other interactions may change the system’s dynamics. Determining for a given nonintrinsically stable network which is which would be important for determining which parts of the network are susceptible to this specific type of attack.

Lastly, as mentioned in the introduction networks are dynamic in two distinct ways. The first is the one considered in this paper, which is the changing state of the network’s elements. The seconding is the evolving structure or topology of the network. As time-varying time-delays effect the network’s structure of interactions these delays also effect the underlying topology of the network. An important implication of this paper is that certain topological changes to a network, e.g. those induced by time-delays, can in general have a destabilizing effect on the network’s dynamics. However, if the network’s dynamics are intrinsically stable, this class of topological transformations does not effect the network’s asymptotic state. It is unknown if there are other types of intrinsic dynamics, i.e. other stronger forms of dynamics, that are resilient to changes in the network’s structure.

8 Appendix

Here we give the proofs of the results found in this paper. We begin by proving the result(s) of Sections 3.

8.1 Appendix A

A proof of Proposition 1 is the following.

Proof.

Let $\mathbf{x}^{*}=[x_{1}^{*},x_{2}^{*},\ldots,x_{n}^{*}]^{T}$ be a fixed point of a dynamical network $(F,X)$. Then by definition

[TABLE]

Let $D$ be a delay distribution with $\max_{ij}d_{ij}\leq L$ for some finite $L>0$. With the usual ordering of the component spaces of $\mathbf{x}\in X_{L}$, we have

[TABLE]

Hence, the extended fixed point $E_{L}(x^{*})$ is a fixed point of $(F_{D},X_{L})$.

8.2 Appendix B

Next we give a proof of Main Result 1, one of the two main results found in Section 4.

Proof.

Let $\mathbf{x},\mathbf{y}\in X$ and $k>0$ be arbitrary. Note that for any $F\in M$ with corresponding $A\in S$ we have, by definition of $A$ being a Lipschitz matrix of $F$, that

[TABLE]

where $\preceq$ denotes an element-wise inequality. Thus, for the specific instance $(\{F^{(k)}\}_{k=1}^{\infty},X)$ given in the hypothesis, we have inductively

[TABLE]

By the definition of the joint spectral radius, there exists some positive constant $C$ (possibly dependant on $\mathbf{x}$ and $\mathbf{y}$) such that

[TABLE]

where $\left\|\mathbf{x}\right\|_{\infty}=\max_{i}\left\lvert x_{i}\right\rvert$.

Now, assume $\mathbf{x}^{*}$ is a shared fixed point of $(F,X)$ for all $F\in M$. Then $\mathscr{F}^{1}(\mathbf{x}^{*})=F^{(1)}(\mathbf{x}^{*})=\mathbf{x}^{*}$, and if it is assumed that $\mathscr{F}^{k-1}(\mathbf{x}^{*})=\mathbf{x}^{*}$, then it follows immediately that

[TABLE]

Hence, by induction, $\mathbf{x}^{*}$ is a fixed point of $(\{F^{(k)}\}_{k=1}^{\infty},X)$. Thus

[TABLE]

so $\lim_{k\to\infty}\mathscr{F}^{k}(\mathbf{x}^{0})=\mathbf{x}^{*}$ for all initial conditions $\mathbf{x}^{0}\in X$. ∎

8.3 Appendix C

Next we prove Main Result 3, which we do by dividing the proof of this result into several lemmata. First, we make explicit the effect of time delays on the Lipschitz matrix of a dynamical network.

Lemma 1**.**

(Structure of the Lipschitz Matrix of a Delayed Network)* Let $(F,X)$ be a dynamical network with Lipschitz matrix $A=[a_{ij}]\in\mathbb{R}^{n\times n}$ and $D=[d_{ij}]\in\mathbb{N}^{n\times n}$ a delay distribution matrix with $\max_{i,j}d_{ij}\leq L$. Let $A_{D}$ be defined in terms of $A$ as*

[TABLE]

where each $A_{\ell}\in\mathbb{R}^{n\times n}$ is defined element-wise as $A_{\ell}=\begin{bmatrix}a_{ij}1_{d_{ij}=\ell}\end{bmatrix}$, with the indicator function $1_{d_{ij}=\ell}$ defined as

[TABLE]

Then $A_{D}$ is a Lipschitz matrix of $(F_{D},X_{L})$.

Proof.

Recall that for $\mathbf{x}\in X_{L}$, we order the components $x_{i,\ell}$ of $\mathbf{x}$ as

[TABLE]

where $x_{i,\ell}\in X_{i,\ell}$ for $i=1,2,\ldots,n$ and $\ell=0,1,\ldots,L$. Let $\mathbf{x},\mathbf{y}\in X$ be given. Then

[TABLE]

which matches the first $n$ rows of $A_{D}$. For $\ell\geq 1$,

[TABLE]

which yields the identity matrices $I_{n}$ in $A_{D}$. ∎

The following theorem follows as a direct corollary of Lemma 3.3 of [10], which is needed in our proof of Main Result 3.

Theorem 3**.**

*Let $(F,X)$ be a dynamical network with Lipschitz matrix $A$. Then for any delay distribution $D$ the constant time-delayed dynamical network $(F_{D},X_{D})$ has the Lipschitz matrix $A_{D}$ with

(i) $\rho(A)\leq\rho(A_{D})<1$ if $\rho(A)<1$;

(ii) $\rho(A_{D})=1$ if $\rho(A)=1$; and

(iii) $\rho(A)\geq\rho(A_{D})>1$ if $\rho(A)>1$.*

We thus have the following immediate corollary by Theorem 3 part (i):

Theorem 4**.**

Suppose $D$ and $\hat{D}$ are delay distribution matrices such that $D\preceq\hat{D}$, i.e. entries of $D$ are less than or equal to the corresponding entries of $\hat{D}$. If $(F,X)$, $(F_{D},X_{D})$, and $(F_{\hat{D}},X_{\hat{D}})$ have the corresponding Lipschitz matrices $A$, $A_{D}$, and $A_{\hat{D}}$, respectively, with $\rho(A)<1$ then

[TABLE]

In other words, the spectral radius of the network is monotonic with respect to the addition of delays if $\rho(A)<1$.

We now require the following results regarding the joint spectral radius. First we need the following definition and theorem originally occurring as Equation (3.1) and Theorem 2 in [27] regarding sets of matrices with independent row uncertainties, respectively.

Definition 9**.**

(Independent Row Uncertainties)* We say that a set of matrices $S\subset\mathbb{R}^{n\times n}$ has independent row uncertainty if $S$ can be expressed as*

[TABLE]

where the sets $Q_{i}\subset\mathbb{R}^{n}$, $1\leq i\leq n$ are closed and bounded.

Theorem 5**.**

(Joint Spectral Radius of Nonnegative Matrices with Independent Row Uncertainty)* Let $S$ be a set of nonnegative matrices with independent row uncertainty. Then*

[TABLE]

Furthermore, we will use the equivalence of Definition 7 of the joint spectral radius with the following representation from [32].

Theorem 6**.**

(Alternate Form of the Joint Spectral Radius)* Given a set of matrices $S\subset\mathbb{R}^{n\times n}$, the joint spectral radius $\rho(S)$ is given by*

[TABLE]

It follows immediately from Theorem 6 that if $S_{1}\subset S_{2}$, then $\rho(S_{1})\leq\rho(S_{2})$. This allows us to give the following proof of Proposition 2.

Proof.

Let $(M,X)$ be a switched network with Lipschitz set $S$. By the form of Theorem 6 we have $\overline{\rho}(S)\leq\overline{\rho}(RI(S))$ since $S\subset RI(S)$. For $1\leq i\leq n$, let $Q_{i}=\{\mathbf{a}_{i}|\text{ }A\in\overline{S}\}$ be the set of $i^{th}$ rows of all $A\in\overline{S}$. Then $RI(S)$ may be expressed as

[TABLE]

implying $RI(S)$ is row-independent. Thus

[TABLE]

by Theorem 5, as desired. ∎

Lemma 2**.**

(Equality of $A_{D}$’s)* Let $L>0$ and $1\leq i\leq n$ be given. Let the matrices $A^{(1)}$, $A^{(2)}$ satisfy $\mathbf{a}^{(1)}_{i}=\mathbf{a}^{(2)}_{i}$, and the matrices $D^{(1)}$, $D^{(2)}$ satisfy $\mathbf{d}^{(1)}_{i}=\mathbf{d}^{(2)}_{i}$. Then $(A^{(1)}_{D^{(1)}})_{i}=(A^{(2)}_{D^{(2)}})_{i}$.*

Proof.

It suffices to show that $(A^{(1)}_{\ell})_{i}=(A^{(2)}_{\ell})_{i}$, where given some $A$ and $D$, $A_{\ell}$ is defined as in Lemma 1. Let $0\leq\ell\leq L$ be arbitrary. Then by Lemma 1,

[TABLE]

Thus $(A^{(1)}_{\ell})_{i}=(A^{(2)}_{\ell})_{i}$, so by Lemma 1, $(A^{(1)}_{D^{(1)}})_{i}=(A^{(2)}_{D^{(2)}})_{i}$. ∎

Lemma 3**.**

(Equality of sets)* Let $(M_{0},X)$ be a switched network with Lipschitz set $S$. Let $L>0$ and $\mathbb{D}=\{D\in\mathbb{N}^{n\times n}|\text{ }\max_{ij}d_{ij}\leq L\}$. Then $RI(\{A_{D}|\text{ }A\in S,D\in\mathbb{D}\})=\{A_{D}|\text{ }A\in RI(S),D\in\mathbb{D}\}$.*

Proof.

Let $(A_{D})^{*}\in RI(\{A_{D}|\text{ }A\in S,D\in\mathbb{D}\})$. Then there exist $(A_{D})^{(1)},\ldots,(A_{D})^{(n)}\in\{A_{D}|\text{ }A\in S,D\in\mathbb{D}\}$ such that $(A_{D})^{*}_{i}=((A_{D})^{(i)})_{i}$. Furthermore, there exist $A^{(1)},\ldots,A^{(n)}\in S$ and $D^{(1)},\ldots,D^{(n)}\in\mathbb{D}$ such that $(A_{D})^{(i)}=A^{(i)}_{D^{(i)}}$. Let $A^{*}$ be constructed as $(A^{*})_{i}=\mathbf{a}^{(i)}_{i}$, and $D^{*}$ be constructed as $(D^{*})_{i}=\mathbf{d}^{(i)}_{i}$. Then $A^{*}\in RI(S)$, and since each $D^{(i)}$ satisfies $d_{ij}\leq L$, we have $D^{*}\in\mathbb{D}$. Thus,

[TABLE]

where the last equality follows from Lemma 2. Therefore, $(A_{D})^{*}=A^{*}_{D^{*}}\in\{A_{D}|\text{ }A\in RI(S),D\in\mathbb{D}\}$, so

[TABLE]

Now let $A_{D}\in\{A_{D}|\text{ }A\in RI(S),D\in\mathbb{D}\}$. Then there exist $A^{(1)},\ldots,A^{(n)}\in S$ and $D\in\mathbb{D}$ such that $(A_{D})_{i}=((A^{(i)})_{D})_{i}$. Let $(A_{D})^{(i)}=(A^{(i)})_{D}$. Then $(A_{D})^{(i)}\in\{A_{D}|\text{ }A\in S,D\in\mathbb{D}\}$ so $A_{D}\in RI(\{A_{D}|\text{ }A\in S,D\in\mathbb{D}\})$. Hence,

[TABLE]

completing the proof. ∎

We now give the proof of Main Result 3 found in Section 5. Following this we show that Main Result 2 is a corollary of this result.

Proof.

Let $(M_{0},X)$ be a switched network with Lipschitz set $S$. Assume $\mathbf{x}^{*}$ is a shared fixed point of $(F,X)$ for all $F\in M_{0}$ and $\rho(A)<1$ for all $A\in RI(S)$. Let $L>0$, $\mathbb{D}=\{D\in\mathbb{N}^{n\times n}|\text{ }\max_{ij}d_{ij}\leq L\}$, $M_{d}=\{F_{D}|F\in M_{0},D\in\mathbb{D}\}$, and let $S_{d}$ be the Lipschitz set of $M_{d}$. We will show $\rho(A_{D})<1$ for all $A_{D}\in RI(S_{d})$ and invoke Proposition 2.

By Lemma 3, we have $RI(S_{d})=\{A_{D}|\text{ }A\in RI(S),D\in\mathbb{D}\}$. Then

[TABLE]

by the hypothesis and Theorem 3. Now given some $A\in RI(S)$, by Lemma 1 and repeated application of Theorem 3 we have that

[TABLE]

Thus by Proposition 2,

[TABLE]

Since $\mathbf{x}^{*}$ is a shared fixed point of $(F,X)$ for all $F\in M_{0}$, by Proposition 1 $E_{L}(\mathbf{x}^{*})$ is a shared fixed point of $(F_{D},X_{L})$ for all $F_{D}\in M_{d}$. Thus by Main Result 1, $E_{L}(\mathbf{x}^{*})$ is a globally attracting fixed point of every instance $(\{F^{(k)}_{D^{(k)}}\}_{k=1}^{\infty},X_{L})$ of $(M_{d},X_{L})$. ∎

Lastly, we give a proof of Main Result 2.

Proof.

Let $M_{0}$ be the singleton set consisting of $F$. Then the Lipschitz set $S$ of $M_{0}$ consists only of the matrix $A$, and so trivially satisfies $S=RI(S)$. Thus the hypothesis of Main Result 3 is trivially satisfied, and the result follows. ∎

9 References