Likelihood-Free Inference with Deep Gaussian Processes

TL;DR

This paper introduces Deep Gaussian Processes as surrogate models in likelihood-free inference, enabling better handling of complex, multimodal distributions and extending Bayesian Optimization's applicability with minimal additional computational cost.

Contribution

It proposes a novel DGP surrogate model that improves likelihood-free inference by effectively managing irregular target distributions, surpassing traditional GPs.

Findings

DGPs outperform GPs on multimodal objective functions

DGPs maintain comparable performance on unimodal distributions

Computational overhead remains negligible for expensive simulators

Abstract

In recent years, surrogate models have been successfully used in likelihood-free inference to decrease the number of simulator evaluations. The current state-of-the-art performance for this task has been achieved by Bayesian Optimization with Gaussian Processes (GPs). While this combination works well for unimodal target distributions, it is restricting the flexibility and applicability of Bayesian Optimization for accelerating likelihood-free inference more generally. We address this problem by proposing a Deep Gaussian Process (DGP) surrogate model that can handle more irregularly behaved target distributions. Our experiments show how DGPs can outperform GPs on objective functions with multimodal distributions and maintain a comparable performance in unimodal cases. This confirms that DGPs as surrogate models can extend the applicability of Bayesian Optimization for likelihood-free inference (BOLFI), while adding computational overhead that remains negligible for computationally intensive simulators.