ADELIA: Automatic Differentiation for Efficient Laplace Inference Approximations

TL;DR

ADELIA introduces an automatic differentiation-based INLA implementation that leverages model sparsity and multi-GPU computing, significantly accelerating gradient computations for large-scale Bayesian inference.

Contribution

It is the first AD-enabled INLA implementation that exploits model structure and GPU parallelism, enabling efficient inference on large models with reduced energy consumption.

Findings

Achieves 4.2–7.9× speedups in gradient computation.

Enables reliable convergence on models with up to 1.9 million latent variables.

Consumes 5–8× less energy than finite difference methods at scale.

Abstract

Spatio-temporal Bayesian inference drives environmental and health sciences using latent Gaussian models. Integrated Nested Laplace Approximations (INLA) enable inference for these models at HPC scale but rely on derivative-based optimization over $d$ hyperparameters. State-of-the-art INLA implementations approximate derivatives via central finite differences (FD), requiring $2d{+}1$ evaluations. These evaluations are embarrassingly parallel, but total work and energy grow with $d$, limiting time-to-solution under fixed budgets. Reverse-mode automatic differentiation (AD) computes exact gradients independently of $d$, but its efficient application to INLA's structured-sparse kernels is an open challenge. We present ADELIA, the first AD-enabled INLA implementation with a structure-exploiting multi-GPU backward pass leveraging model sparsity. We evaluate ADELIA on ten benchmark models, including real-world air-pollution monitoring. We achieve $4.2$--$7.9\times$ per-gradient speedups and reliable convergence on production-scale models with up to 1.9M latent variables, where FD struggles. Even when scaled to 16--32 GPUs to match ADELIA's wall-clock time, FD consumes $5$--$8\times$ more energy.