Effect of leakage delay on Hopf bifurcation in a fractional BAM neural network
Jiazhe Lin, Rui Xu, Liangchen Li, Xiaohong Tian
TL;DR
This paper investigates how leakage delay influences Hopf bifurcation in a fractional BAM neural network, highlighting the critical role of delay in neural dynamics and employing sensitivity analysis to quantify its impact.
Contribution
It introduces a novel analysis of leakage delay effects on Hopf bifurcation in fractional neural networks, combining bifurcation theory with sensitivity assessment methods.
Findings
Hopf bifurcation occurs at critical delay values
Leakage delay significantly affects bifurcation amplitude
Numerical examples confirm theoretical predictions
Abstract
Recently, the influence of leakage delay on the dynamics of integer-order neural networks has been investigated extensively. It has been confirmed that fractional calculus can depict the memory and hereditary attributes of neural networks more accurately. In this paper, we study the existence of Hopf bifurcation in a six-neuron fractional bidirectional associative memory (BAM) neural network with leakage delay. By selecting two appropriate bifurcation parameters and analyzing corresponding characteristic equations, it is verified that the delayed fractional neural network generates a Hopf bifurcation when the bifurcation parameters pass through some critical values. In order to measure how much impact of leakage delay on Hopf bifurcation, sensitivity analysis methods, such as scatter plots and partial rank correlation coefficients (PRCCs), are introduced to assess the sensitivity of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Effect of leakage delay on Hopf bifurcation in a fractional BAM neural network
Jiazhe Lina, Rui Xub,c,, Liangchen Lia, Xiaohong Tianb,c
a Institute of Applied Mathematics, Army Engineering University
Shijiazhuang 050003, Hebei, P.R. China
b Complex Systems Research Center, Shanxi University
Taiyuan 030006, Shanxi, P.R. China
c Shanxi Key Laboratory of Mathematical Techniques and Big Data Analysis on
Disease Control and Prevention, Shanxi University
Taiyuan 030006, Shanxi, P.R. China Author for Correspondence, e-mail: [email protected], [email protected].
Abstract: Recently, the influence of leakage delay on the dynamics of integer-order neural networks has been investigated extensively. It has been confirmed that fractional calculus can depict the memory and hereditary attributes of neural networks more accurately. In this paper, we study the existence of Hopf bifurcation in a six-neuron fractional bidirectional associative memory (BAM) neural network with leakage delay. By selecting two appropriate bifurcation parameters and analyzing corresponding characteristic equations, it is verified that the delayed fractional neural network generates a Hopf bifurcation when the bifurcation parameters pass through some critical values. In order to measure how much impact of leakage delay on Hopf bifurcation, sensitivity analysis methods, such as scatter plots and partial rank correlation coefficients (PRCCs), are introduced to assess the sensitivity of bifurcation amplitudes to leakage delay. Numerical examples are carried out to illustrate the theoretical results and help us gain an insight into the effect of leakage delay vividly.
Keywords: Leakage delay; Hopf bifurcation; fractional order; BAM neural network; sensitivity analysis.
1 Introduction
In the past few decades, BAM neural networks as well as their various generalizations have attracted the attention of many researchers due to their potential applications in parallel computation, associative memory and nonlinear optimization problems (see, e.g. [1, 2, 3, 4, 5, 6, 7, 8], and the references cited therein). The BAM neural networks are originally proposed by Kosko [9], which are composed of neurons arranged in two layers and described by the following system
[TABLE]
where , are the connection weights at the time through the neurons in two layers: the I-layer and the J-layer, and describe the stability of internal neuron processes on the I-layer and the J-layer, respectively. On the I-layer, the neurons whose states are denoted by receive the inputs and the inputs outputted by those neurons in the J-layer via activation functions , while on the J-layer, the neurons whose associated states denoted by receive the inputs and the inputs outputted by those neurons in the I-layer via activation functions .
Obviously, the influence of time delays on the dynamics of BAM neural networks has been not taken into account in system (1.1). Due to the finite speed of signal transmission and amplifiers switching, time delay inevitably exists in neural networks. On one hand, time delays are harmful to the dynamical behavior of considered neural networks, causing oscillation, even chaos. For instance, Xu et al. [2] investigated a six-neuron BAM neural network model with communication delays and illustrated that the model undergoes a Hopf bifurcation in some cases. On the other hand, a proper time delay is advantageous for the dynamical behavior of neural networks, such as, it can improve the stability performance of networks (see, e.g. [10, 11, 12]). Meanwhile, the leakage term in each of the right sides of (1.1) corresponds to a stabilizing negative feedback of the system which acts instantaneously without time delay. In practice, time is required to isolate the static state. Gopalsamy [13] found that time delays in leakage terms have an important influence on the dynamic behavior, and the stability of the neural networks was determined by leakage delays. The mathematical model is described by
[TABLE]
where and denote the leakage delay, respectively; and represent the communication delay, respectively. Similarly, Tian and Xu [14] illustrated that time delays in the stabilizing negative feedback terms will have a tendency to destabilize a system.
In recent years, fraction-order derivatives provide an excellent tool for the description of memory and hereditary properties of various materials and processes. In general, plenty of practical objects can be described clearly by the fractional differential equations, due to their more degrees of freedom and infinite memory. Hence, the research of dynamical analysis of fractional neural networks has gained a lot of attention and some valuable results have been referred to [15, 17, 16, 18, 19]. As we know, the integer-order calculus can only determine the local features of neural networks, while the fractional calculus can depict the memory and hereditary attributes more accurately [20, 21].
In [15], Huang et al. studied the stability and bifurcation of four-neuron fractional BAM neural networks with time delay in leakage terms and illustrated that the leakage delay has a destabilizing influence on the stability performance, which cannot be ignored. Furthermore, Huang and Cao [16] concentrated on the issue of bifurcation analysis for high-order fractional BAM neural networks involving leakage delay and obtain similar bifurcation analysis results. In [18], Tao et al. proposed a fractional two-gene regulatory network model with delays and found that the delayed fractional genetic network can generate a Hopf bifurcation when the total delay passes through some critical values.
However, it is noteworthy that some of above results about stability and bifurcation for the fractional systems are defective on the analysis methods or relatively few neurons in both two layers in BAM neural networks. When the number of neurons in BAM neural network (1.2) is larger, the simplified system (1.3) can reflect the really large neural networks more closely. Motivated by Xu et al. [2] and Huang et al. [15, 16], the primary objective of this paper is to study the existence of Hopf bifurcation for the following six-neuron fractional BAM neural network involving leakage delay
[TABLE]
where is the fractional order, , stand for state variables of the th neuron in I-layer and J-layer, respectively; describes the stability of internal neuron processes on the I-layer and the J-layer, respectively; , represent connection weights; , denote activation functions; is leakage delay, is communication delay.
Above discussion may raise the question that, for a given fractional BAM neural network, which one contributes more to the influence of dynamical behavior, leakage delay or communication delay? In order to assess how much impact of leakage delay on Hopf bifurcation, we introduce sensitivity analysis into this paper. Sensitivity analysis (SA) assesses how variations in model outputs can be apportioned, qualitatively or quantitatively, to different input sources [22, 23]. Through the process of recalculating outcomes under alternative assumptions, we can determine the impact of a variable by scatter plots and PRCCs. Sensitivity analysis provides an evaluation of how much each input is contributing to the output uncertainty and performs the role of ordering by importance, namely, by the strength and relevance of the inputs in determining the variations in the output [24].
The main contributions of this paper can be summarized as follows:
The problem of stability and bifurcation in a six-neuron fractional BAM neural network with leakage delay is discussed by introducing two appropriate bifurcation parameters, and the existence of Hopf bifurcation for the proposed network is established.
The impact of the order on the critical frequencies and bifurcation points is further demonstrated numerically for the fractional BAM neural network.
Sensitivity analysis is introduced into bifurcation analysis of the fractional BAM neural networks, which helps us to estimate whether leakage delay or communication delay contributes more to the influence of dynamical behavior for a given fractional BAM neural network.
The outline of the paper is listed as follows: In Section 2, the stability of the equilibrium and the existence of Hopf bifurcation are studied. In Section 3, numerical simulations are carried out to illustrate the validity of the main results. Sensitivity analysis of leakage delay and communication delay is performed in Section 4. A brief remark is given in Section 5 to conclude this work.
2 Main results for bifurcation analysis
In this section, we introduce two appropriate bifurcation parameters. Through analyzing corresponding characteristic equations, we obtain the local stability of the equilibrium and the existence of Hopf bifurcation to system (1.3). Before bifurcation analysis, we address the following assumption.
(H1) , , , for .
There are several definitions of fractional derivatives. Riemann-Liouville definition and Caputo definition are commonly used. Since Caputo derivative only requires initial conditions given by means of integer-order derivative, representing well-understood features of physical situation and making it more applicable to real world problems [15, 16]. Hence, the definition of Caputo derivative is adopted in this paper.
Definition 2.1**.**
[25]* For a function , Caputo fractional derivative of order is defined by*
[TABLE]
where , and is a positive integer such that .
Moreover, when ,
[TABLE]
It is obvious that the origin is an equilibrium point of system (1.3) under Assumption (H1). Linearizing system (1.3) at the origin, we obtain that
[TABLE]
where , . By applying Laplace transformation, the characteristic matrix of system (2.1) is
[TABLE]
Thus, we obtain the characteristic equation of system (2.1) as follows
[TABLE]
where positive constants , , , , , , , , , , , , , , , , , are defined in Appendix A.
Define , which are selected as bifurcation parameters. If and , Eq. (2.2) is changed into
[TABLE]
where
[TABLE]
The following lemma about the stability of fractional autonomous systems is listed for later analysis.
Lemma 2.1**.**
[26, 27]* The following autonomous system*
[TABLE]
where , , , this system is asymptotically stable if and only if . In this case, each component of the states decays towards [math] like . Also, this system is stable if and only if and those critical eigenvalues that satisfy have geometric multiplicity one.
Theorem 2.1**.**
System (1.3) is asymptotically stable when and hold, where is defined as follows
[TABLE]
Proof.
If holds, it follows that all the roots satisfy . According to Lemma 2.1, we can easily conclude that system (1.3) is asymptotically stable when . ∎
2.1 Hopf bifurcation with respect to
If and , i.e., , Eq. (2.2) becomes
[TABLE]
Multiplying on both sides of Eq. (2.3) and denoting , it follows that
[TABLE]
Define the six roots of Eq. (2.4) as , where and are the real and imaginary parts of , respectively. Note that
[TABLE]
To find possible periodic solutions, which may bifurcate from a Hopf bifurcation point, let be a root of (2.5). Substituting into (2.5) and separating the real and imaginary parts yields
[TABLE]
From (2.6), it follows that
[TABLE]
Noting that , we obtain that
[TABLE]
Define the bifurcation point of system (1.3) as follows:
[TABLE]
To establish the main results of this section, we make the following assumptions.
(H2) Eq. (2.7a) has no positive real root.
(H3) Eq. (2.7a) has at least one positive real root.
(H4) , where , are defined in Appendix B.
Lemma 2.2**.**
Let , be the root of Eq. (2.3) near satisfying , , then the following transversality condition holds
[TABLE]
Proof.
Based on implicit function theorem, we calculate the derivative of Eq. (2.3) with respect to as follows
[TABLE]
Direct calculation shows that
[TABLE]
where
[TABLE]
The real part of at is
[TABLE]
where and are the real and imaginary parts of , respectively; and are the real and imaginary parts of , respectively. Based on Assumption (H4), the transversality condition meets. This completes the proof. ∎
According to Assumptions (H1)-(H4), the following theorem can be derived.
Theorem 2.2**.**
For system (1.3), the following results hold
(i)* If Assumptions (H1), (H2) are satisfied, the zero equilibrium point is global asymptotically stable for .*
(ii)* If Assumptions (H1), (H3), (H4) and Lemma 2.2 hold,*
a)* The zero equilibrium point is locally asymptotically stable for ;*
b)* system (1.3) undergoes a Hopf bifurcation at the origin when , i.e., it has a branch of periodic solutions bifurcating from the zero equilibrium point near .*
2.2 Hopf bifurcation with respect to
If , fix and the characteristic equation (2.2) becomes
[TABLE]
where
[TABLE]
Let and substitute it into Eq. (2.8), it follows that
[TABLE]
in which , , . Separating the real and imaginary parts, we obtain that
[TABLE]
Squaring both sides of the two equations of (2.10), respectively, and adding them up yields
[TABLE]
Noting that , we consider the two cases:
(1) If , then Eq. (2.11) takes the following form:
[TABLE]
It is easy to see that Eq. (2.12) is equivalent to
[TABLE]
where
[TABLE]
Denote , Eq. (2.13) is changed into
[TABLE]
From Ferrari’s method by back changing the variables and using the formulas for the quadratic and cubic equations, the four roots for Eq. (2.14) are listed as follows
[TABLE]
where
[TABLE]
in which
[TABLE]
with , . The bifurcation point of system (1.3) in this case follows that
[TABLE]
(2) If , then (2.11) is changed into
[TABLE]
Direct calculation shows that Eq. (2.17) has similar roots with Eq. (2.12). Define the bifurcation point of system (1.3) as , , where is defined by (2.16). By the above discussion, we obtain that is a function with respect to . Denote . From (2.11), we have
[TABLE]
Similarly, we make the following assumptions.
(H5) Eq. (2.18) has at least one positive real root.
(H6) , where , are defined in Appendix C.
Lemma 2.3**.**
Let , be the root of Eq. (2.8) near satisfying , , then the following transversality condition holds
[TABLE]
Proof.
Based on implicit function theorem, we calculate the derivative of Eq. (2.8) with respect to as follows
[TABLE]
Direct calculation yields
[TABLE]
where
[TABLE]
The real part of at is
[TABLE]
where and are the real and imaginary parts of , respectively; and are the real and imaginary parts of , respectively. From Assumption (H6), the transversality condition meets. This completes the proof. ∎
Theorem 2.3**.**
For system (1.3), if Assumptions (H1), (H5), (H6) and Lemma 2.3 hold, we obtain the following results
a)* The zero equilibrium point is locally asymptotically stable for ;*
b)* system (1.3) undergoes a Hopf bifurcation at the origin when , i.e., it has a branch of periodic solutions bifurcating from the zero equilibrium point near .*
3 Representative examples
In this section, we present some numerical simulations of system (1.3) to verify the analytical predictions obtained in Section 2. The simulation results are based on Adams-Bashforth-Moulton algorithm [28] and step-length . Selecting appropriate system parameters, system (1.3) is changed into
[TABLE]
where activation functions are chosen as .
In the next two subsections, we will illustrate our theoretical results by selecting different leakage delays and communication delays. Example 1 is relevant for Section 3.1, while Example 2 is related to Section 3.2. By means of plotting temporal solutions of , , , , and versus and corresponding phase diagrams of system (3.1) projected on different quadrants, the dynamical behavior of system (3.1) can be depicted clearly.
3.1 Example 1
In this example, we choose , i.e., , and illustrate the theoretical results obtained in Section 3.1. The fractional order here is chosen as and the initial value is selected as . From (2.7), we obtain that the critical frequency and the bifurcation point . From Fig.1 and Fig.2, it can be illustrated that the zero equilibrium point is locally asymptotically stable when , while, Fig.3 and Fig.4 demonstrate that the zero equilibrium point loses stability and Hopf bifurcation occurs when . This example indicates that leakage delay and communication delay indeed have a destabilizing influence on the stability performance of system (3.1).
3.2 Example 2
In this example, we fix and expound the theoretical results obtained in Section 3.2. The fractional order here is chosen as and the initial value is selected as . From (2.15), (2.16) and (2.18), we approximately calculate the bifurcation point . In
Fig.5 and Fig.6, we observe that the zero equilibrium point is locally asymptotically stable when , while, Fig.7 and Fig.8 indicate that the zero equilibrium point lose stability and Hopf bifurcation occurs when . Because , communication delay only is not strong enough to destabilize system (3.1). But as increases from to , namely, leakage delay increases from to , system (3.1) occurs a Hopf bifurcation, which illustrates the destabilizing impact of leakage delay directly.
Besides, to reveal the effects of fractional order on the critical frequencies and bifurcation points, we use the system parameters which are the same to those in Example 1. Fig.9 implies that, when the fractional order increases from to , the critical frequency
declines and the scale of decrease becomes smaller accordingly. As for the bifurcation point , it decreases and reach the minimal value at . Later, it increases to the maximum value . It is worth mentioning that when is near , a little leakage delay or communication delay may lead to the occurrence of Hopf bifurcation. According to the association between the fractional order and the bifurcation point, one can delay the onset of an inherent bifurcation even eliminate the existence of Hopf bifurcation by changing the fractional order, which is insightful for the design and applications of neural networks.
4 Sensitivity analysis of leakage delay
Latin hypercube sampling (LHS) belongs to Monte Carlo class of sampling methods, and was introduced by Mckay et al. [29]. LHS allows an un-biased estimate of the average model output, with the advantage that it requires fewer samples than simple random sampling to achieve the same accuracy [29]. LHS is a so-called stratified sampling without replacement technique, where the random parameter distributions are divided into equal probability intervals, which are then sampled. represents the sample size. The choice for should be at least , where is the number of varied parameters, but usually much larger to ensure accuracy [30].
Scatter plot is a simple but useful tool to give a direct visual indication of sensitivity after sampling the model over its input distributions [31]. We set the sample size and suppose that leakage delay and communication delay are subject to uniform distribution on . Other system parameters are the same to Fig.3 and Fig.4. Using Adams-Bashforth-Moulton algorithm, we obtain the stable Hopf bifurcation amplitude for each sample. Finally, we obtain Hopf bifurcation amplitudes of the six neurons and describe it in forms of scatter plots. Fig.10 and Fig.11 show the scatter plots of the amplitudes of , , , , and with respect to and , respectively, which imply that and are both positively correlative variables with the amplitudes of six-neuron states. However, how much strength and relevance between the delays and the amplitudes is not clear.
Next, we will study PRCCs of the amplitudes of , , , , and to and . Marino et al. [30] mentioned that PRCCs provide a measure of the strength of a linear association between the inputs and the outputs. The positive or negative of PRCCs respectively denote the positive or negative correlation with and , and the sizes of PRCCs (the height of the bar graph in Fig.12) measure the strength of the correlation. As we can see in Fig.12, and are positively correlative variables to the amplitudes. Specially, we observe that the PRCCs of are smaller than the PRCCs of , which implies that the impact of leakage delay is limited compared with communication delay in system (3.1). Furthermore, we observe that the PRCCs of and are both more than , which means that the dynamical behavior of system (3.1) is mainly determined by the leakage delay and the communication delay.
Remark 4.1**.**
The PRCCs in Fig.12 implies that leakage delay indeed affects the dynamical behavior of the proposed network, but the influence of leakage delay may be not strong enough in contrast with communication delay, which only holds in some specific situations (see, e.g., system (3.1)). It is worth mentioning that PRCCs method provides a specific tool to assess different delays which can affect the dynamical behavior of fractional neural networks.
5 Discussion and conclusion
In this paper, we proposed a fractional six-neuron BAM neural network and established the existence of Hopf bifurcation. It was verified that the delayed fractional neural network generates a Hopf bifurcation when the bifurcation parameter or passes through some critical values. By numerical simulations, we demonstrated the effectiveness of our theoretical results and found that leakage delay weakens the stability of the fractional BAM neural network, which is similar to the works in [15, 16]. We also investigated the influence of the order on the critical frequencies and bifurcation points numerically. Through sensitivity analysis, we clearly observed that how much impact of leakage delay and communication delay on the dynamical behavior of system (3.1).
In [15], Huang et al. investigated a four-neuron fractional BAM neural network and revealed the effects of the leakage delay on the bifurcation points. For the sake of convenience, Huang et al. assumed that leakage delay equals to communication delay, which is similar to our theoretical analysis in Section 3.1, but didn’t consider the general situation, namely, leakage delay differs from communication delay. Besides, in [15], the influence of leakage delay was illustrated by comparing two fractional BAM neural network with and without leakage delay, which cannot clearly indicate the impact. As for the really large neural networks, the association between the fractional order and the bifurcation points is more complicated and our simulations may reflect the really fact more closely in compare with [15].
Besides, neural networks are realized by electronic circuit, and the density of electromagnetic field is generally not uniform. Therefore, in factual modeling, only considering the change of time seems to be not comprehensive when electrons are moving in asymmetric and nonuniform electromagnetic fields. Thus, in our further work, we will introduce the diffusion effect into the fractional BAM neural networks.
Appendix A
[TABLE]
[TABLE]
[TABLE]
Appendix B
[TABLE]
[TABLE]
Appendix C
[TABLE]
[TABLE]
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Z. Zhang, G. Peng, D. Zhou, Periodic solution to Cohen-Grossberg BAM neural networks with delays on time scales, Journal of the Franklin Institute 348 (2011) 2759-2781.
- 2[2] C. Xu, X. Tang, M. Liao, Stability and bifurcation analysis of a six-neuron BAM neural network model with discrete delays, Neurocomputing 74 (2011) 689-707.
- 3[3] G. Bao, Z. Zeng, Analysis and design of associative memories based on recurrent neural network with discontinuous activation functions, Neurocomputing 77 (2012) 101-107.
- 4[4] B. Liu, Global exponential stability for BAM neural networks with time-varying delays in the leakage terms, Nonlinear Analysis: Real World Applications 14 (2013) 559-566.
- 5[5] M. Liu, X. Xu, C. Zhang, Stability and global Hopf bifurcation for neutral BAM neural network, Neurocomputing 145 (2014) 122-130.
- 6[6] C. Xu, Q. Zhang, Existence and global exponential stability of anti-periodic solutions for BAM neural networks with inertial term and delay, Neurocomputing 153 (2015) 108-116.
- 7[7] F. Wang, Y. Yang, X. Xu, et al., Global asymptotic stability of impulsive fractional-order BAM neural networks with time delay, Neural Computing & Applications 28 (2017) 345-352.
- 8[8] W. Yang, W. Yu, J. Cao, Global exponential stability and lag synchronization for delayed memristive fuzzy Cohen-Grossberg BAM neural networks with impulses, Neural Networks 98 (2018) 122-153.