Approximate Bayesian Computation with Statistical Distances for Model Selection

TL;DR

This paper systematically evaluates discrepancy-based Approximate Bayesian Computation (ABC) methods for model selection, comparing their empirical performance and calibration across various simulation scenarios and real-world applications.

Contribution

It provides a comprehensive comparison of full data ABC methods using statistical distances with traditional summary-statistic-based ABC, highlighting their strengths and limitations.

Findings

Full data ABC with Wasserstein, Creamer-von-Mises, and MMD can produce stable, well-calibrated posterior model probabilities.

Performance of discrepancy-based ABC varies with model overlap and dependence, sometimes degrading in complex scenarios.

Application to toad movement models demonstrates practical implications and guidance for likelihood-free model choice.

Abstract

Model selection in the presence of intractable likelihoods remains a central challenge in Bayesian inference. Approximate Bayesian computation (ABC) provides a flexible likelihood-free framework, but its use for model choice is known to be sensitive to the choice of summary statistics, often leading to poorly calibrated posterior model probabilities. Recent ABC variants based on statistical distances allow comparisons to be performed directly on empirical distributions, avoiding data reduction and offering improved theoretical guarantees under suitable conditions. This paper provides a systematic evaluation of discrepancy-based ABC methods for Bayesian model selection, focusing on their empirical behavior across a range of simulation settings and levels of model complexity. We compare full data ABC approaches based on Wasserstein, Creamer-von-Mises, and maximum mean discrepancy metrics with summary-statistic-based ABC and neural network classifiers. The results highlight settings in which full data ABC yields stable and well-calibrated posterior model probabilities, as well as scenarios where performance degrades due to model overlap or dependence. An application to toad movement models illustrates the practical implications of these findings. Overall, the study clarifies the strengths and limitations of discrepancy-based ABC for likelihood-free model choice and provides guidance for its use in realistic inferential settings.