Perspective from the Literature on the Role of Expert Judgment in Scientific and Statistical Research and Practice

TL;DR

This paper reviews the literature on the critical role of expert judgment in statistical inference and scientific research, emphasizing its importance in hypothesis testing, Bayesian modeling, and interpretation of results.

Contribution

It synthesizes existing literature to highlight the role of expertise in statistics and proposes recommendations for improving judgment use in scientific practice.

Findings

Expert judgment is vital in hypothesis testing and Bayesian analysis.

Proper interpretation of results depends on expert judgment.

Recommendations can enhance the effective use of judgment in statistics.

Abstract

This article, produced as a result of the Symposium on Statistical Inference, is an introduction to the literature on the function of expertise, judgment, and choice in the practice of statistics and scientific research. In particular, expert judgment plays a critical role in conducting Frequentist hypothesis tests and Bayesian models, especially in selection of appropriate prior distributions for model parameters. The subtlety of interpreting results is also discussed. Finally, external recommendations are collected for how to more effectively encourage proper use of judgment in statistics. The paper synthesizes the literature for the purpose of creating a single reference and inciting more productive discussions on how to improve the future of statistics and science.