Estimating the Value of Evidence-Based Decision Making

TL;DR

This paper presents an empirical framework using Bayesian methods to quantify the value of evidence-based decision making, helping organizations optimize evidence gathering and decision strategies for better outcomes.

Contribution

It introduces a novel empirical approach to estimate the value of evidence-based decision making and assess return on investment in statistical precision.

Findings

Evidence-based decision making's value varies with decision rules.

Significance-based rules can underutilize or negatively impact value.

The framework accounts for heterogeneity in parameters.

Abstract

In an era of data abundance, statistical evidence is increasingly critical for business and policy decisions. Yet, organizations lack empirical tools to assess the value of evidence-based decision making (EBDM), optimize statistical precision, and balance the costs of evidence-gathering strategies against their benefits. To tackle these challenges, this article introduces an empirical framework to estimate the value of EBDM and evaluate the return on investment in statistical precision and project ideation. The framework leverages parametric and nonparametric empirical Bayes methods to account for parameter heterogeneity and measure how statistical precision changes the value of evidence. The value extracted from statistical evidence depends critically on how organizations translate evidence into policy decisions. Commonly used decision rules based on statistical significance can leave substantial value unrealized and, in some cases, generate negative expected value.