Likelihood-free Model Choice for Simulator-based Models with the Jensen--Shannon Divergence

TL;DR

This paper introduces JSD-Razor, a new consistent model scoring criterion for simulator-based models without likelihood functions, leveraging Jensen--Shannon divergence's asymptotic properties and demonstrating its effectiveness through synthetic and real data examples.

Contribution

The paper develops JSD-Razor, a novel likelihood-free model choice criterion based on Jensen--Shannon divergence, addressing limitations of existing methods like Bayes factors.

Findings

JSD-Razor is consistent for model selection in likelihood-free settings.

It outperforms traditional criteria in synthetic experiments.

It shows favorable properties in real-world modeling examples.

Abstract

Choice of appropriate structure and parametric dimension of a model in the light of data has a rich history in statistical research, where the first seminal approaches were developed in 1970s, such as the Akaike's and Schwarz's model scoring criteria that were inspired by information theory and embodied the rationale called Occam's razor. After those pioneering works, model choice was quickly established as its own field of research, gaining considerable attention in both computer science and statistics. However, to date, there have been limited attempts to derive scoring criteria for simulator-based models lacking a likelihood expression. Bayes factors have been considered for such models, but arguments have been put both for and against use of them and around issues related to their consistency. Here we use the asymptotic properties of Jensen--Shannon divergence (JSD) to derive a consistent model scoring criterion for the likelihood-free setting called JSD-Razor. Relationships of JSD-Razor with established scoring criteria for the likelihood-based approach are analyzed and we demonstrate the favorable properties of our criterion using both synthetic and real modeling examples.