The Future of Indirect Evidence

TL;DR

This paper discusses the evolving role of indirect evidence in statistical analysis, highlighting the shift from traditional objectivity towards methods like Empirical Bayes to better utilize large-scale data sets.

Contribution

It introduces the importance of indirect evidence in modern statistics and explores Empirical Bayes as a promising approach to integrate direct and indirect data.

Findings

Empirical Bayes offers a practical compromise between direct and indirect evidence.

Modern data sets favor a less rigid standard of statistical objectivity.

There is a trend towards incorporating more indirect evidence in statistical practice.

Abstract

Familiar statistical tests and estimates are obtained by the direct observation of cases of interest: a clinical trial of a new drug, for instance, will compare the drug's effects on a relevant set of patients and controls. Sometimes, though, indirect evidence may be temptingly available, perhaps the results of previous trials on closely related drugs. Very roughly speaking, the difference between direct and indirect statistical evidence marks the boundary between frequentist and Bayesian thinking. Twentieth-century statistical practice focused heavily on direct evidence, on the grounds of superior objectivity. Now, however, new scientific devices such as microarrays routinely produce enormous data sets involving thousands of related situations, where indirect evidence seems too important to ignore. Empirical Bayes methodology offers an attractive direct/indirect compromise. There is already some evidence of a shift toward a less rigid standard of statistical objectivity that allows better use of indirect evidence. This article is basically the text of a recent talk featuring some examples from current practice, with a little bit of futuristic speculation.