False (and Missed) Discoveries in Financial Economics

TL;DR

This paper introduces a new calibration method for statistical errors in finance, using a double-bootstrap approach to control false discoveries and misses, revealing current methods' limited power in detecting outperformers.

Contribution

It proposes a novel calibration technique for Type I and II errors in financial testing, incorporating a double-bootstrap method to set specific error thresholds.

Findings

Current methods lack power to detect outperforming managers

New calibration approach controls false discovery rate effectively

Double-bootstrap method establishes error thresholds with practical implications

Abstract

Multiple testing plagues many important questions in finance such as fund and factor selection. We propose a new way to calibrate both Type I and Type II errors. Next, using a double-bootstrap method, we establish a t-statistic hurdle that is associated with a specific false discovery rate (e.g., 5%). We also establish a hurdle that is associated with a certain acceptable ratio of misses to false discoveries (Type II error scaled by Type I error), which effectively allows for differential costs of the two types of mistakes. Evaluating current methods, we find that they lack power to detect outperforming managers.