Hedging in Sequential Experiments

TL;DR

This paper introduces a game-theoretic framework for experimentalists to hedge against the risk of null results and false positives, using financial and portfolio theories to manage experimental uncertainty.

Contribution

It develops a novel method to hedge experimental risk by capitalizing test martingales and constructing derivative instruments within a game-theoretic setting.

Findings

Test martingale wealth can be priced using risk-neutral valuation.

A portfolio combining risky and risk-free assets remains a test martingale.

Hedging instruments can be derived from the test martingale process.

Abstract

Experimentation involves risk. The investigator expends time and money in the pursuit of data that supports a hypothesis. In the end, the investigator may find that all of these costs were for naught and the data fail to reject the null. Furthermore, the investigator may not be able to test other hypotheses with the same data set in order to avoid false positives due to p-hacking. Therefore, there is a need for a mechanism for investigators to hedge the risk of financial and statistical bankruptcy in the business of experimentation. In this work, we build on the game-theoretic statistics framework to enable an investigator to hedge their bets against the null hypothesis and thus avoid ruin. First, we describe a method by which the investigator's test martingale wealth process can be capitalized by solving for the risk-neutral price. Then, we show that a portfolio that comprises the risky test martingale and a risk-free process is still a test martingale which enables the investigator to select a particular risk-return position using Markowitz portfolio theory. Finally, we show that a function that is derivative of the test martingale process can be constructed and used as a hedging instrument by the investigator or as a speculative instrument by a risk-seeking investor who wants to participate in the potential returns of the uncertain experiment wealth process. Together, these instruments enable an investigator to hedge the risk of ruin and they enable a investigator to efficiently hedge experimental risk.