A Pragmatic Framework for Bayesian Utility Magnitude-Based Decisions

TL;DR

This paper introduces a pragmatic Bayesian decision-making framework that calculates a unified expected utility score using a tangible effect scale, aiding practical decisions by integrating statistical evidence with value considerations.

Contribution

It presents a novel, unified, non-arbitrary points scale for effect magnitudes and a comprehensive utility-based decision framework combining Bayesian inference with practical value trade-offs.

Findings

Framework produces a single expected utility score for decision-making.

Inclusion of individual response variability characterizes benefits and harms.

Accessible spreadsheet implementation facilitates practical application.

Abstract

This article presents a pragmatic framework for making formal, utility-based decisions from statistical inferences. The method calculates an expected utility score for an intervention by combining Bayesian posterior probabilities of different effect magnitudes with points representing their practical value. A key innovation is a unified, non-arbitrary points scale (1-9 for small to extremely large) derived from a principle linking tangible outcomes across different effect types. This tangible scale enables a principled "trade-off" method for including values for loss aversion, side effects, and implementation cost. The framework produces a single, definitive expected utility score, and the initial decision is made by comparing the magnitude of this single score to a user-defined smallest important net benefit, a direct and intuitive comparison made possible by the scale's tangible nature. This expected utility decision is interpreted alongside clinical magnitude-based decision probabilities or credible interval coverage to assess evidence strength. Inclusion of a standard deviation representing individual responses to an intervention (or differences between settings with meta-analytic data) allows characterization of differences between individuals (or settings) in the utility score expressed as proportions expected to experience benefit, a negligible effect, and harm. These proportions provide context for the final decision about implementation. Users must perform sensitivity analyses to investigate the effects of systematic bias and of the subjective inputs on the final decision. This framework, implemented in an accessible spreadsheet, has not been empirically validated. It represents a tool in development, designed for practical decision-making from available statistical evidence and structured thinking about values of outcomes.