Scalable marginalization of correlated latent variables with applications to learning particle interaction kernels

TL;DR

This paper develops scalable methods for marginalizing correlated latent variables, especially in Gaussian process models and particle interaction kernels, enabling efficient inference in complex spatio-temporal data.

Contribution

It introduces a novel marginalization technique for interaction kernels using sparse covariance representations and conjugate gradient, enhancing scalability and accuracy.

Findings

Efficient marginalization of correlated data using Kalman filter and smoother.

Extension of scalable marginalization to multivariate and spatio-temporal models.

Improved predictive accuracy and computational efficiency in particle trajectory forecasting.

Abstract

Marginalization of latent variables or nuisance parameters is a fundamental aspect of Bayesian inference and uncertainty quantification. In this work, we focus on scalable marginalization of latent variables in modeling correlated data, such as spatio-temporal or functional observations. We first introduce Gaussian processes (GPs) for modeling correlated data and highlight the computational challenge, where the computational complexity increases cubically fast along with the number of observations. We then review the connection between the state space model and GPs with Mat{\'e}rn covariance for temporal inputs. The Kalman filter and Rauch-Tung-Striebel smoother were introduced as a scalable marginalization technique for computing the likelihood and making predictions of GPs without approximation. We then introduce recent efforts on extending the scalable marginalization idea to the linear model of coregionalization for multivariate correlated output and spatio-temporal observations. In the final part of this work, we introduce a novel marginalization technique to estimate interaction kernels and forecast particle trajectories. The achievement lies in the sparse representation of covariance function, then applying conjugate gradient for solving the computational challenges and improving predictive accuracy. The computational advances achieved in this work outline a wide range of applications in molecular dynamic simulation, cellular migration, and agent-based models.