Inference on high-dimensional implicit dynamic models using a guided intermediate resampling filter

TL;DR

This paper introduces a novel particle filtering method designed for high-dimensional, nonlinear, non-Gaussian models with intractable transition densities, improving scalability and applicability to complex systems.

Contribution

The authors develop a guided intermediate resampling filter that enhances particle filter scalability for implicit models with intractable transition densities, applicable to continuous-time latent processes.

Findings

Successfully applied to stochastic Lorenz 96 model.

Effective in modeling infectious disease dynamics.

Improves inference in high-dimensional nonlinear systems.

Abstract

We propose a method for inference on moderately high-dimensional, nonlinear, non-Gaussian, partially observed Markov process models for which the transition density is not analytically tractable. Markov processes with intractable transition densities arise in models defined implicitly by simulation algorithms. Widely used particle filter methods are applicable to nonlinear, non-Gaussian models but suffer from the curse of dimensionality. Improved scalability is provided by ensemble Kalman filter methods, but these are inappropriate for highly nonlinear and non-Gaussian models. We propose a particle filter method having improved practical and theoretical scalability with respect to the model dimension. This method is applicable to implicitly defined models having analytically intractable transition densities. Our method is developed based on the assumption that the latent process is defined in continuous time and that a simulator of this latent process is available. In this method, particles are propagated at intermediate time intervals between observations and are resampled based on a forecast likelihood of future observations. We combine this particle filter with parameter estimation methodology to enable likelihood-based inference for highly nonlinear spatiotemporal systems. We demonstrate our methodology on a stochastic Lorenz 96 model and a model for the population dynamics of infectious diseases in a network of linked regions.