Embed and Emulate: Contrastive representations for simulation-based inference

TL;DR

This paper presents Embed and Emulate (E&E), a contrastive learning-based method for efficient simulation-based inference in high-dimensional, complex systems, enabling accurate parameter estimation without costly simulations.

Contribution

E&E introduces a novel contrastive learning approach to learn low-dimensional embeddings and emulators, improving high-dimensional SBI performance over existing methods.

Findings

E&E effectively handles high-dimensional data and complex posteriors.

Theoretical analysis confirms properties of the learned latent space.

E&E outperforms existing SBI methods in a chaotic Lorenz 96 system.

Abstract

Scientific modeling and engineering applications rely heavily on parameter estimation methods to fit physical models and calibrate numerical simulations using real-world measurements. In the absence of analytic statistical models with tractable likelihoods, modern simulation-based inference (SBI) methods first use a numerical simulator to generate a dataset of parameters and simulated outputs. This dataset is then used to approximate the likelihood and estimate the system parameters given observation data. Several SBI methods employ machine learning emulators to accelerate data generation and parameter estimation. However, applying these approaches to high-dimensional physical systems remains challenging due to the cost and complexity of training high-dimensional emulators. This paper introduces Embed and Emulate (E&E): a new SBI method based on contrastive learning that efficiently handles high-dimensional data and complex, multimodal parameter posteriors. E&E learns a low-dimensional latent embedding of the data (i.e., a summary statistic) and a corresponding fast emulator in the latent space, eliminating the need to run expensive simulations or a high dimensional emulator during inference. We illustrate the theoretical properties of the learned latent space through a synthetic experiment and demonstrate superior performance over existing methods in a realistic, non-identifiable parameter estimation task using the high-dimensional, chaotic Lorenz 96 system.