Strategic Hypothesis Testing

TL;DR

This paper models the strategic interaction between a principal and an agent in hypothesis testing, analyzing how the principal can set optimal p-value thresholds considering the agent's incentives, with empirical validation in drug approval data.

Contribution

It introduces a game-theoretic model of hypothesis testing with strategic agents, providing a clear characterization of optimal p-value thresholds and error behaviors.

Findings

Principal's errors show monotonic behavior relative to p-value thresholds.

Optimal p-value threshold can be characterized by an efficiently computable critical value.

Model validated using real-world drug approval data.

Abstract

We examine hypothesis testing within a principal-agent framework, where a strategic agent, holding private beliefs about the effectiveness of a product, submits data to a principal who decides on approval. The principal employs a hypothesis testing rule, aiming to pick a p-value threshold that balances false positives and false negatives while anticipating the agent's incentive to maximize expected profitability. Building on prior work, we develop a game-theoretic model that captures how the agent's participation and reporting behavior respond to the principal's statistical decision rule. Despite the complexity of the interaction, we show that the principal's errors exhibit clear monotonic behavior when segmented by an efficiently computable critical p-value threshold, leading to an interpretable characterization of their optimal p-value threshold. We empirically validate our model and these insights using publicly available data on drug approvals. Overall, our work offers a comprehensive perspective on strategic interactions within the hypothesis testing framework, providing technical and regulatory insights.