perms: Likelihood-free estimation of marginal likelihoods for binary response data in Python and R

TL;DR

Perms is a likelihood-free software package in Python and R that accurately estimates marginal likelihoods for binary response models using permutation counting, applicable to both parametric and nonparametric models.

Contribution

The paper introduces perms, a computationally efficient package implementing permutation counting for marginal likelihood estimation in binary response models, including nonparametric approaches.

Findings

Perms provides accurate ML estimates without likelihood calculations.

The package handles large datasets efficiently.

It is applicable to both parametric and nonparametric models.

Abstract

In Bayesian statistics, the marginal likelihood (ML) is the key ingredient needed for model comparison and model averaging. Unfortunately, estimating MLs accurately is notoriously difficult, especially for models where posterior simulation is not possible. Recently, Christensen (2023) introduced the concept of permutation counting, which can accurately estimate MLs of models for exchangeable binary responses. Such data arise in a multitude of statistical problems, including binary classification, bioassay and sensitivity testing. Permutation counting is entirely likelihood-free and works for any model from which a random sample can be generated, including nonparametric models. Here we present perms, a package implementing permutation counting. As a result of extensive optimisation efforts, perms is computationally efficient and able to handle large data problems. It is available as both an R package and a Python library. A broad gallery of examples illustrating its usage is provided, which includes both standard parametric binary classification and novel applications of nonparametric models, such as changepoint analysis. We also cover the details of the implementation of perms and illustrate its computational speed via a simple simulation study.