# Information geometry for approximate Bayesian computation

## TL;DR

This paper applies information geometry and relative entropy to analyze and improve the Approximate Bayesian Computation (ABC) algorithm, providing theoretical insights and practical guidelines for its performance and sensitivity.

## Contribution

It introduces an information-theoretic framework for analyzing ABC, including bias, error bounds, and sensitivity, which aids in optimizing the algorithm's efficiency.

## Key findings

- Relative entropy analysis clarifies ABC behavior as a function of thresholds and data size.
- The framework identifies important directions for reducing computational costs.
- Simulation results validate theoretical error bounds and sensitivity analysis.

## Abstract

The goal of this paper is to explore the basic Approximate Bayesian Computation (ABC) algorithm via the lens of information theory. ABC is a widely used algorithm in cases where the likelihood of the data is hard to work with or intractable, but one can simulate from it. We use relative entropy ideas to analyze the behavior of the algorithm as a function of the threshold parameter and of the size of the data. Relative entropy here is data driven as it depends on the values of the observed statistics. Relative entropy also allows us to explore the effect of the distance metric and sets up a mathematical framework for sensitivity analysis allowing to find important directions which could lead to lower computational cost of the algorithm for the same level of accuracy. In addition, we also investigate the bias of the estimators for generic observables as a function of both the threshold parameters and the size of the data. Our analysis provides error bounds on performance for positive tolerances and finite sample sizes. Simulation studies complement and illustrate the theoretical results.