Concentration of discrepancy-based approximate Bayesian computation via Rademacher complexity

TL;DR

This paper develops a unified theoretical framework for discrepancy-based approximate Bayesian computation (ABC) using Rademacher complexity, providing explicit concentration bounds and extending analysis to non-i.i.d. and misspecified models.

Contribution

It introduces Rademacher complexity into the analysis of summary-free ABC, enabling uniform, discrepancy-agnostic, and verifiable asymptotic results under broad conditions.

Findings

Provides explicit concentration bounds for ABC posteriors.

Extends theoretical analysis to non-i.i.d. and misspecified models.

Includes analysis of Wasserstein distance and MMD within the framework.

Abstract

There has been increasing interest on summary-free solutions for approximate Bayesian computation (ABC) which replace distances among summaries with discrepancies between the empirical distributions of the observed data and the synthetic samples generated under the proposed parameter values. The success of these strategies has motivated theoretical studies on the limiting properties of the induced posteriors. However, there is still the lack of a theoretical framework for summary-free ABC that (i) is unified, instead of discrepancy-specific, (ii) does not require to constrain the analysis to data generating processes and statistical models meeting specific regularity conditions, but rather facilitates the derivation of limiting properties that hold uniformly, and (iii) relies on verifiable assumptions that provide explicit concentration bounds clarifying which factors govern the limiting behavior of the ABC posterior. We address this gap via a novel theoretical framework that introduces the concept of Rademacher complexity in the analysis of the limiting properties for discrepancy-based ABC posteriors, including in non-i.i.d. and misspecified settings. This yields a unified theory that relies on constructive arguments and provides more informative asymptotic results and uniform concentration bounds, even in settings not covered by current studies. These advancements are obtained by relating the asymptotic properties of summary-free ABC posteriors to the behavior of the Rademacher complexity associated with the chosen discrepancy in the family of integral probability semimetrics (IPS). The IPS class extends summary-based distances, and includes the Wasserstein distance and maximum mean discrepancy, among others. As clarified in specialized theoretical analyses of popular IPS discrepancies and via illustrative simulations, this perspective improves the understanding of summary-free ABC.