Comparing and weighting imperfect models using D-probabilities

TL;DR

This paper introduces D-probabilities, a divergence-based method for weighting models that improves model comparison and aggregation by addressing limitations of Bayesian model probabilities.

Contribution

The paper presents D-probabilities, a novel divergence-based approach for model weighting that avoids prior sensitivity and promotes model diversity.

Findings

D-probabilities effectively compare and weight imperfect models.

The method automatically penalizes model complexity.

Applications demonstrate improved model selection and aggregation.

Abstract

We propose a new approach for assigning weights to models using a divergence-based method ({\em D-probabilities}), relying on evaluating parametric models relative to a nonparametric Bayesian reference using Kullback-Leibler divergence. D-probabilities are useful in goodness-of-fit assessments, in comparing imperfect models, and in providing model weights to be used in model aggregation. D-probabilities avoid some of the disadvantages of Bayesian model probabilities, such as large sensitivity to prior choice, and tend to place higher weight on a greater diversity of models. In an application to linear model selection against a Gaussian process reference, we provide simple analytic forms for routine implementation and show that D-probabilities automatically penalize model complexity. Some asymptotic properties are described, and we provide interesting probabilistic interpretations of the proposed model weights. The framework is illustrated through simulation examples and an ozone data application.