Critical speeding up as an early warning signal of regime switching
Mathew Titus, Zach Gelbaum, James Watson

TL;DR
This paper introduces 'critical speeding up,' a phenomenon where decreasing variance and autocorrelation signal an imminent regime shift due to basin compression, contrasting with traditional critical slowing down indicators.
Contribution
It identifies and demonstrates a new warning sign, critical speeding up, in systems undergoing regime change, expanding early warning signals beyond critical slowing down.
Findings
Critical speeding up occurs when basin of attraction compresses.
Variance and autocorrelation decrease before regime shifts.
Both rising and falling statistical signals can indicate imminent change.
Abstract
The use of critical slowing down as an early warning indicator for regime switching in observations from stochastic environments and noisy dynamical models has been widely studied and implemented in recent years. Some systems, however, have been shown to avoid critical slowing down prior to a transition between equilibria, e.g. (Ditlevsen and Johnsen, 2010). Possible explanations include non-smooth potential driving the dynamic (Hastings and Wysham, 2010) or large perturbations driving the system out of the initial basin of attraction. In this paper we discuss a phenomenon analogous to critical slowing down, where a slow parameter change leads to a high likelihood of a regime shift and creates signature warning signs in the statistics of the process's sample paths. In short, if a basin of attraction is compressed under a parameter change then the potential well steepens, leading to a…
Click any figure to enlarge with its caption.
Figure 1
Figure 2Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFault Detection and Control Systems
Critical speeding up as an early warning signal of regime switching.
Mathew Titus1,2, Zach Gelbaum1,2, James R. Watson1
Abstract
The use of critical slowing down as an early warning indicator for regime switching in observations from stochastic environments and noisy dynamical models has been widely studied and implemented in recent years. Some systems, however, have been shown to avoid critical slowing down prior to a transition between equilibria, e.g. (Ditlevsen and Johnsen, 2010). Possible explanations include non-smooth potential driving the dynamic (Hastings and Wysham, 2010) or large perturbations driving the system out of the initial basin of attraction. In this paper we discuss a phenomenon analogous to critical slowing down, where a slow parameter change leads to a high likelihood of a regime shift and creates signature warning signs in the statistics of the process’s sample paths. In short, if a basin of attraction is compressed under a parameter change then the potential well steepens, leading to a drop in the time series’ variance and autocorrelation; precisely the opposite warning signs exhibited by critical slowing down. This effect, which we call “critical speeding up,” is demonstrated using a simple ecological model exhibiting an Allee effect. The fact that both dropping and rising variance / autocorrelation can indicate imminent state change should underline the need for reliable modeling of any empirical system where one desires to forecast regime change.
1College of Earth, Ocean and Atmospheric Sciences, Oregon State University, Corvallis, Oregon, USA
2The Prediction Lab, Corvallis, Oregon, USA
Keywords: early warning signals, regime change, critical slowing down, critical speeding up, complex systems
When studying time series data for dynamical systems which exhibit critical transitions, that is sudden changes in equilibrium behavior, a widely used early warning signal for an oncoming transition is critical slowing down (CSD) (Scheffer et al., 2009; Dakos et al., 2012). This signature for the system being at risk of a large transition is based on the theory of stochastic dynamical systems (Kuehn, 2013), but has been observed in a variety of empirical tests, both in nature and in the laboratory (see (Carpenter and Brock, 2006; van Belzen et al., 2017), (Wen et al., 2018) and references therein). At its core, CSD assumes that the stochastic process experiences a smooth potential which is varying slowly in time, where is some random process. If the potential nears a bifurcation point, such as a pitchfork or fold bifurcation, then the shape of around its equilibrium necessarily flattens out as its minimum (stable point) becomes a degenerate critical point. The lessening or loss of local curvature, responsible for the mean-reverting property within the basin of attraction, means that excursions of away from its stable point grow both in extent and length of time. The concomitant increase in variance and autocorrelation of the sample path of is what we refer to as critical slowing down. We expand on this phenomenon in the next section.
The catalogued examples where CSD is observed are often carefully controlled laboratory settings (Kramer and Ross, 1985) or models of natural systems subject to relatively small perturbations (Carpenter and Brock, 2006; Dakos and Bascompte, 2014), but many empirical observations of dynamical systems fail to exhibit critical slowing down prior to making a change of regime (Ditlevsen and Johnsen, 2010; Hastings and Wysham, 2010; Rozek et al., 2017). That critical state transitions appear to commonly occur in nature away from a bifurcation in the underlying governing dynamic speaks to the value in devising early warning indicators for other ‘high-risk’ situations where a regime shift may be becoming increasingly likely. This lack of CSD prior to transition has been discussed in the literature recently, with explanations including large exogenous perturbations(Boettiger and Hastings, 2013; Ditlevsen and Johnsen, 2010) or the potential lacking smoothness (Hastings and Wysham, 2010); these translate, respectively, to a sudden change in the governing equation (a fundamental change in the statistics of ) or a departure from the classical setting of modeling the dynamics with a system of partial differential equations and perturbing the smooth dynamical model with the noise source .
In the present paper we study an alternative culprit which retains both smoothness of the potential and the (quasi-)stationarity of the system. We assume the usual setting for CSD: a dynamical system governed by a slowly-changing potential , and a fixed noise term, for some and a brownian motion. If experiences a bifurcation the degeneracy of will create a slowing effect, increasing both variance and autocorrelation. However, we consider instead potentials where the long-term evolution of narrows the basin of attraction that the process is initiated within, avoiding bifurcations. Given a potential well with fixed height but shrinking width leads to a ‘rattling’ effect, wherein the stabilizing gradient grows, punishing excursions away from the stable point (minimum of ) and shortening their extent. This manifests itself in decreased variance and decreased autocorrelation of the sample path of or as we call it “rattle”. These signs indicate that a stochastic transition is becoming more likely, giving us an early warning of a stochastic transition, rather than a transition compelled by bifurcation. (Note that this is not at odds with the conclusions of (Boettiger and Hastings, 2013), for the transition is not purely due to noise.)
We further show that if the narrowing of the potential is uniform (a linear rescaling of the spatial axis) then the effect on the exit time distribution is equivalent to rescaling the time variable. That is, in terms of the risk of escaping the basin of attraction within the next units of time, narrowing the potential is the same as speeding up the process’s evolution, and from this standpoint it is clear that such a transformation increases the chance of regime switching. This is the motivation for our term “critical speeding up” (CSU), as the time until a critical transition is shortened.
In the next section we give a simple mathematical description of both critical slowing down and critical speeding up. As an example of CSU occurring in a simple ecological model, Section 2 introduces a population-level model for a species with an Allee effect (i.e. when individual fitness and/or the population growth rate is a function of population density). We assume that for this species that the population growth rate is proportional to the population density; then under a shrinking habitat the system begins to rattle (see Figure 1) and exhibit CSU effects as risk of extinction grows. In the final section we discuss the broader implications of this effect, in particular that the results presented here stress the need to implement a faithful model before relying on either CSD or CSU as an early warning signal.
1 Mathematical background: CSD and CSU
Suppose for simplicity that we are dealing with a one-dimensional process where may be a population size, or a concentration of a chemical, or a fraction of people who subscribe to a given belief, at time . The systems we are interested in allow to be modeled by a stochastic differential equation (SDE) as it generally obeys some dynamical law, but is subject to exogenous perturbations or uncertainty in measurement:
[TABLE]
Here is the (smooth) drift function, describing the deterministic differential equation would obey if ; the noise is modeled by the product of the coefficient and , a one-dimensional brownian motion. Since we are working in one-dimension, a potential function exists
[TABLE]
so that (1) becomes .
If the process is initiated in a basin of attraction which, over time, is bounded on the left (right) by (respectively, ), i.e. , we write for the exit time of the process from that basin:
[TABLE]
In the sequel we will use the potential
[TABLE]
as a testbed for both phenomena.
1.1 Equivalence of narrowing and time change
Suppose the process obeys the SDE , then if the spatial scale is compressed by a factor so that the potential defining the dynamic becomes , then (leaving the diffusion term unchanged) we have a second process, , which evolves under the influence of the narrowed potential , and so experiences at point a drift :
[TABLE]
Let us define . Then from (5) and the identity we have
[TABLE]
and so taking we find obeys , the same equation as .
Since obeys the same dynamic as , the probability of exiting the basin by some time (when initiated from ) is equal to . Define and . Then finally we have the following relation between exit time distributions:
[TABLE]
Accordingly, contracting space by a factor leads to the same exit time statistics (i.e. critical transition rates) as contracting time by a factor of . Effectively, a uniform narrowing of can be interpreted as speeding up the evolution of , increasing the rate of extreme events.
We note that the above argument also applies for , in which case the potential widens, and has the same statistics as a copy of the process which evolves a factor of slower; this slowing effect is the origin of critical slowing down.
1.2 CSD and CSU
In this subsection we discuss the fundamentals of CSD; it is not intended to be an exhaustive treatment and readers should turn to the literature for full details and generality (Kuehn, 2013). As above, we assume that the process of interest is one-dimensional and described by the SDE , where is smooth, is a positive constant, and is a brownian motion. As we are not interested in deriving the explicit estimates of CSD using normal forms and slow-fast systems theory (Berglund and Gentz, 2006; Kuehn, 2013), but rather the qualititative features, we approximate the potentials with the lowest order term in their Taylor expansions. This simplifies the dynamics to those of the Ornstein-Uhlenbeck processes or brownian motions with drift. We also assume that the process originates within a basin of attraction, , and that the (smooth) functions are defined so that the basin of attraction is given by for all future times that it exists.
Near the unique minimum of within the basin the second-order approximation of can be used in place of the true potential to define a process whose dynamics are close to those of while it remains near equilibrium. Since we are interested in the system (4), we consider the modified equation
[TABLE]
which simply follows from the Taylor expansion of about the point .
As mentioned above, this process is an Ornstein-Uhlenbeck (OU) process so its characteristics are well-known. In particular, the -autocorrelation function defined by
[TABLE]
is given for the Ornstein-Uhlenbeck process by the formula
[TABLE]
From this it is immediate that the variance is given by
[TABLE]
and so we have for (7) that the autocorrelation with is given by
[TABLE]
Now, if we allow to change in time so that it slowly approaches zero from above, a bifurcation occurs at . Let us define ; then will approach 0 slowly enough that CSD is observable, and we see that the variance grows linearly in time, for , and similarly for the autocorrelation.
On the other hand, if we do not approach the bifurcation point, fixing in time, and instead take to be increasing in time, say and restrict to , then, again translating the potential’s minimum to the origin, we have
[TABLE]
demonstrating the speeding up incurred by narrowing the potential by a factor .
We may again apply (8) to recover the variance and autocorrelation. One sees that for this process
[TABLE]
and likewise the 1-step autocorrelation decays like . While we have restricted our attention to a rather specific example, this decay of autocorrelation and variance, which we call critical speeding up, is characteristic of contracting potentials.
2 Observation of CSU in a population model
Here we introduce a population model that exhibits CSU before a critical state transition (population collapse). We assume a strong Allee effect, proportional to the territory size; this could model a species which has displaced its competitors, benefitting from a cooperative advantage against another species. We also assume that the reproduction rate is a function of population density, rather than total population. These features create an attracting region for the fixed point at zero, so that after passing below a critical threshold the species will become extinct (discounting a restoring perturbation).
This model is described by the following equation
[TABLE]
where all parameters and the population size are assumed to be nonnegative.
We interpret as the maximum carrying capacity of the species, say if they control of their potential territory, while is the carrying capacity when a fraction is controlled. The product represents the strength of the Allee effect; suppose for example that the population only controls one quarter of the contested territory () rather than half (), then the Allee effect is weakened as their competitors have control of a greater swath of the environment, reducing the degree to which the individuals are able to benefit from cooperation. Finally, is chosen so that a population of size reproduces at a rate when distributed over the entire territory. When restricted to a fraction of the total land, the population density increases by a factor of . Hence, the reproductive rate is given by .
As a final addition, we assume the model originates at its positive equilibrium () and add the noise term to model exogenous perturbations (change in population through fatal accidents, immigration from another distant population, twins, etc.), and write for the population process:
[TABLE]
We study this model’s vulnerability to extinction events under the increasing stress of an encroaching competing species, which decreases the available territory, driving towards zero:
[TABLE]
In Figure 1 we see the effect of slowly decreasing over 5000 independent trials; the lower panel shows the distribution of the time of collapse, which we define as the first time the system’s population drops below , as this separates the two dynamical regimes (extinction and survival). For all of these trials we fix , , and . Also pictured in Figure 1 are the variance and autocorrelation functions (calculated over a rolling window of length 10), after averaging over all sample paths which have yet to exit the basin of attraction. The hallmarks of CSU are evident in the time series; as drops, the equilibrium points (, , and ) gather toward to origin, and both the variance and autocorrelation of the series are reduced. Despite the variance and autocorrelation dropping by over 50%, none of the 5000 trials survived past 871 units of time (surviving sample path not pictured).
As another verification of the theory in subsection 1.1, we also run repeated simulations of starting from for equal to a fixed value between 0.2 and 1.2 (ten equally distributed values were used, making for 20,000 independent trials). Statistics of the results are plotted in Figure 2. It is shown that the time for the population to exit , the basin of attraction about , decreases with decreasing as expected from the previous analysis, i.e. the risk of a critical transition grows as the potential narrows. However, the calculation of the covariance and autocorrelation (see Figure 2) show that the system becomes more brittle as drops towards , and critical speeding up gives the precursor signal for population collapse. Note that all values are plotted on a semi-log scale.
3 Discussion
Using stochastic dynamical systems theory we have shown that noisy dynamical systems such as seen in nature, can not only fail to exhibit critical slowing down prior to a regime shift, they may actually exhibit a speeding up of their dynamic, with a decrease in both variance and autocorrelation. To a practitioner observing a system with a narrowing potential, a naive application of the theory of CSD would suggest the system is stabilizing as it begins to rattle, despite the impending critical transition. We used an example model of population growth to demonstrate this in practice, showing that a population experiencing territory loss and an Allee effect exhibits critical speeding up prior to collapse. We hope that this study underscores the necessity of having high-fidelity descriptions of the system dynamics before applying an early warning indicator such as CSD or CSU, a sentiment expressed also in (Boettiger and Hastings, 2012).
One might wonder just how pathological a system must be for it to compress a potential well under a slow parameter shift without significantly decreasing the depth of the well (leading to a bifurcation and CSD), nor increasing the depth of the well (restoring stability, despite the onset of CSU). We acknowledge that if the well deepens, the signal of CSU will be a false positive. However, this is analogous to a potential well broadening while deepening which leads to false positives under the CSD paradigm. Both signals offer an early warning under the appropriate conditions, but neither is a panacea for predicting regime change. Only with prior knowledge of the space of all possible evolutions of the underlying potential, , such as in the toy model (4), can we begin to map sample path statistics to high or low risk states with confidence.
As a second example of time series statistics belying the approach of a transition, one can imagine the model system (4) from Section 1 evolving such that both and . This would create both CSD and CSU effects in the time series. As these signals interfere with one another, simply observing the autocorrelation and variance of the time series may fail to find either a slowing down or a speeding up of the time series. This is another example of abrupt transitions occurring without a precursor slowing down, and may be responsible for some of the examples in the literature of critical transitions without CSD such as the regime switching in (Ditlevsen and Johnsen, 2010). At the least, this should call to question the idea that using CSD as an early warning indicator with no assumptions or prior knowledge of the system studied, a practice promoted elsewhere in the literature.
There are several reasons that we expect to find systems in nature that exhibit CSU. First, the model above describes a population governed by three simple rules, so it would be no surprise to find such a population within the earth’s ecology. Further, suppose a sequence of failures must occur for a complex system (e.g. a power grid, social network or a full food-web) to experience catastrophe. If its subsystems become correlated, then the probability of a submodule failing (say, a generator shutting down, individuals disbanding, multiple species going extinct etc.) may be lowered due to the individuals’ pooled resilience, but the number of failures required is lowered as well. In this case, the probability of an excursion of the state from stability () to instability () may be the same, but the number of events necessary (i.e. the distance in state space) has been reduced. This sort of thresholding between stability and instability is not uncommon in man-made systems, see for example (Reason, 1995).
As a third example, many engineered systems obey stochastic differential-algebraic equations (SDAEs), wherein the noisy dynamical system is required to satisfy some algebraic constraints (Schein and Denk, 1998), so that for some function we have
[TABLE]
Forcing the solution to remain near the zero set of gives an overdetermined solution set; when solutions exist they are often “stiff,” meaning that the solution has a high-frequency component that can lead to sudden large-scale change in . As the solution is subject to the condition , we may reinterpret the system as having a strong restoring potential such that solutions to minimize , so that obeys an equation of the form . Of course, the strength of suggests that is naturally quite narrow about its minima.
When studying a complex system with a variety of possible stable states, the notion of CSU may be useful in determining the likelihood of the system occupying a given equilibrium. Despite stabilizing drift that may be quite strong, if the basin of attraction about the stable point is prone to narrowing the system may not inhabit the basin for long periods of time. This principle may be useful in studying the time-dynamics and evolution of protein-folding, food-webs, or social structures (Zwanzig, 1997; Ma et al., 2017; Nekovee et al., 2007).
Finally, we mention that the principle of critical speeding up - that spatial compression of a basin of attraction can lead to regime switching as well as diminished autocorrelation and variance - holds in higher dimensions. One can also allow for systems which have stable limit cycles rather than equilibrium points. We do, however, relegate the proof of this fact to the larger literature on SDEs and future work.
Acknowledgments
The author would like to acknowledge support from the DARPA YFA project N66001-17-1-4038 and George Hagstrom for suggesting the ecological model used here.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Ditlevsen and Johnsen (2010) Ditlevsen, P. D.; Johnsen, S. J. Geophysical Research Letters 2010 , 37 .
- 2Hastings and Wysham (2010) Hastings, A.; Wysham, D. B. Ecology Letters 2010 , 13 , 464–472.
- 3Scheffer et al. (2009) Scheffer, M.; Bascompte, J.; Brock, W. A.; Brovkin, V.; Carpenter, S. R.; Dakos, V.; Held, H.; Van Nes, E. H.; Rietkerk, M.; Sugihara, G. Nature 2009 , 461 , 53.
- 4Dakos et al. (2012) Dakos, V.; Carpenter, S. R.; Brock, W. A.; Ellison, A. M.; Guttal, V.; Ives, A. R.; Kéfi, S.; Livina, V.; Seekell, D. A.; van Nes, E. H.; Scheffer, M. PLOS ONE 2012 , 7 , 1–20.
- 5Kuehn (2013) Kuehn, C. Journal of Nonlinear Science 2013 , 23 , 457–510.
- 6Carpenter and Brock (2006) Carpenter, S. R.; Brock, W. A. Ecology Letters 2006 , 9 , 311–318.
- 7van Belzen et al. (2017) van Belzen, J.; van de Koppel, J.; Kirwan, M. L.; van der Wal, D.; Herman, P. M. J.; Dakos, V.; Kéfi, S.; Scheffer, M.; Guntenspergen, G. R.; Bouma, T. J. Nature Communications 2017 , 8 , 15811 EP –.
- 8Wen et al. (2018) Wen, H.; Ciamarra, M. P.; Cheong, S. A. Plo S one 2018 , 13 , e 0191439.
