Credal Prediction based on Relative Likelihood

TL;DR

This paper introduces a new credal prediction method based on relative likelihood, enabling better uncertainty representation by defining credal sets through plausible models, with demonstrated effectiveness on benchmark datasets.

Contribution

It presents a theoretically grounded approach to credal prediction using relative likelihood and modifies ensemble techniques for practical approximation.

Findings

Superior uncertainty representation demonstrated

Maintains predictive performance

Outperforms several state-of-the-art baselines

Abstract

Predictions in the form of sets of probability distributions, so-called credal sets, provide a suitable means to represent a learner's epistemic uncertainty. In this paper, we propose a theoretically grounded approach to credal prediction based on the statistical notion of relative likelihood: The target of prediction is the set of all (conditional) probability distributions produced by the collection of plausible models, namely those models whose relative likelihood exceeds a specified threshold. This threshold has an intuitive interpretation and allows for controlling the trade-off between correctness and precision of credal predictions. We tackle the problem of approximating credal sets defined in this way by means of suitably modified ensemble learning techniques. To validate our approach, we illustrate its effectiveness by experiments on benchmark datasets demonstrating superior uncertainty representation without compromising predictive performance. We also compare our method against several state-of-the-art baselines in credal prediction.