PAVI: Plate-Amortized Variational Inference

TL;DR

This paper introduces PAVI, a scalable variational inference method that leverages plate amortization to efficiently handle massive latent spaces in large-scale hierarchical models, demonstrated on neuroimaging data.

Contribution

The paper proposes plate amortization, a novel approach to sharing parameters across i.i.d. variables, enabling efficient and scalable variational inference for large population studies.

Findings

Achieves orders of magnitude faster training times.

Handles models with up to a million latent parameters.

Demonstrates effectiveness on neuroimaging data.

Abstract

Given some observed data and a probabilistic generative model, Bayesian inference aims at obtaining the distribution of a model's latent parameters that could have yielded the data. This task is challenging for large population studies where thousands of measurements are performed over a cohort of hundreds of subjects, resulting in a massive latent parameter space. This large cardinality renders off-the-shelf Variational Inference (VI) computationally impractical. In this work, we design structured VI families that can efficiently tackle large population studies. To this end, our main idea is to share the parameterization and learning across the different i.i.d. variables in a generative model -symbolized by the model's plates. We name this concept plate amortization, and illustrate the powerful synergies it entitles, resulting in expressive, parsimoniously parameterized and orders of magnitude faster to train large scale hierarchical variational distributions. We illustrate the practical utility of PAVI through a challenging Neuroimaging example featuring a million latent parameters, demonstrating a significant step towards scalable and expressive Variational Inference.