Evaluating Bayesian Model Visualisations

TL;DR

This paper presents a protocol and software framework for quantitatively evaluating Bayesian model visualisations, aiming to improve user comprehension and decision-making under uncertainty.

Contribution

It introduces a standardized evaluation protocol and software tools for Bayesian visualisations, along with a user study on interactivity effects.

Findings

Interactivity in visualisations can enhance understanding.

The evaluation protocol supports reproducibility and comparison.

Design guidelines for effective Bayesian visualisations are proposed.

Abstract

Probabilistic models inform an increasingly broad range of business and policy decisions ultimately made by people. Recent algorithmic, computational, and software framework development progress facilitate the proliferation of Bayesian probabilistic models, which characterise unobserved parameters by their joint distribution instead of point estimates. While they can empower decision makers to explore complex queries and to perform what-if-style conditioning in theory, suitable visualisations and interactive tools are needed to maximise users' comprehension and rational decision making under uncertainty. In this paper, propose a protocol for quantitative evaluation of Bayesian model visualisations and introduce a software framework implementing this protocol to support standardisation in evaluation practice and facilitate reproducibility. We illustrate the evaluation and analysis workflow on a user study that explores whether making Boxplots and Hypothetical Outcome Plots interactive can increase comprehension or rationality and conclude with design guidelines for researchers looking to conduct similar studies in the future.