Bayesian Ensemble Echo State Networks for Enhancing Binary Stochastic Cellular Automata

TL;DR

This paper introduces a novel Bayesian ensemble echo state network approach to improve modeling of binary spatio-temporal data governed by stochastic cellular automata, incorporating latent dynamics and reservoir ensembles for enhanced predictive power.

Contribution

It pioneers the integration of echo state networks as latent processes within a Bayesian framework and introduces ensemble reservoirs, advancing both CA modeling and ESN methodologies.

Findings

Effective modeling of binary spatio-temporal data with unknown local rules.

Improved predictive accuracy on simulated and real-world datasets.

Demonstrated applicability to fire spread and rabies transmission data.

Abstract

Binary spatio-temporal data are common in many application areas. Such data can be considered from many perspectives, including via deterministic or stochastic cellular automata, where local rules govern the transition probabilities that describe the evolution of the 0 and 1 states across space and time. One implementation of a stochastic cellular automata for such data is with a spatio-temporal generalized linear model (or mixed model), with the local rule covariates being included in the transformed mean response. However, in real world applications, we seldom have a complete understanding of the local rules and it is helpful to augment the transformed linear predictor with a latent spatio-temporal dynamic process. Here, we demonstrate for the first time that an echo state network (ESN) latent process can be used to enhance the local rule covariates. We implement this in a hierarchical Bayesian framework with regularized horseshoe priors on the ESN output weight matrices, which extends the ESN literature as well. Finally, we gain added expressiveness from the ESNs by considering an ensemble of ESN reservoirs, which we accommodate through model averaging. This is also new to the ESN literature. We demonstrate our methodology on a simulated process in which we assume we do not know all of the local CA rules, as well as a fire evolution data set, and data describing the spread of raccoon rabies in Connecticut, USA.