Sharp Results for Hypothesis Testing with Risk-Sensitive Agents

TL;DR

This paper develops a game-theoretic framework for hypothesis testing involving strategic data sources, providing sharp bounds on false discovery rates and analyzing the impact of agents' utilities on testing protocols.

Contribution

It introduces a novel analysis of hypothesis testing with strategic agents, deriving optimal bounds on false discovery rates and demonstrating the maximin property of testing protocols.

Findings

Derived an upper bound on the Bayes false discovery rate for strategic agents.

Showed how agents' opt-in decisions reveal bounds on their prior null probabilities.

Demonstrated the maximin property of the proposed testing protocols.

Abstract

Statistical protocols are often used for decision-making involving multiple parties, each with their own incentives, private information, and ability to influence the distributional properties of the data. We study a game-theoretic version of hypothesis testing in which a statistician, also known as a principal, interacts with strategic agents that can generate data. The statistician seeks to design a testing protocol with controlled error, while the data-generating agents, guided by their utility and prior information, choose whether or not to opt in based on expected utility maximization. This strategic behavior affects the data observed by the statistician and, consequently, the associated testing error. We analyze this problem for general concave and monotonic utility functions and prove an upper bound on the Bayes false discovery rate (FDR). Underlying this bound is a form of prior elicitation: we show how an agent's choice to opt in implies a certain upper bound on their prior null probability. Our FDR bound is unimprovable in a strong sense, achieving equality at a single point for an individual agent and at any countable number of points for a population of agents. We also demonstrate that our testing protocols exhibit a desirable maximin property when the principal's utility is considered. To illustrate the qualitative predictions of our theory, we examine the effects of risk aversion, reward stochasticity, and signal-to-noise ratio, as well as the implications for the Food and Drug Administration's testing protocols.