Standing on the Shoulders of Machine Learning: Can We Improve Hypothesis Testing?

TL;DR

This paper integrates modern machine learning classification models into hypothesis testing, enabling more flexible, powerful, and context-aware tests, demonstrated through improved unit root testing in time series econometrics.

Contribution

It introduces the use of machine learning algorithms as mapping functions for hypothesis testing, creating pseudo-composite tests and improving power and accuracy.

Findings

Boosted decision stumps recover the full size-power trade-off.

Complex algorithms like random forests enable multi-statistic testing.

Application to unit root testing shows 17% accuracy and 36% sensitivity improvements.

Abstract

In this paper we have updated the hypothesis testing framework by drawing upon modern computational power and classification models from machine learning. We show that a simple classification algorithm such as a boosted decision stump can be used to fully recover the full size-power trade-off for any single test statistic. This recovery implies an equivalence, under certain conditions, between the basic building block of modern machine learning and hypothesis testing. Second, we show that more complex algorithms such as the random forest and gradient boosted machine can serve as mapping functions in place of the traditional null distribution. This allows for multiple test statistics and other information to be evaluated simultaneously and thus form a pseudo-composite hypothesis test. Moreover, we show how practitioners can make explicit the relative costs of Type I and Type II errors to contextualize the test into a specific decision framework. To illustrate this approach we revisit the case of testing for unit roots, a difficult problem in time series econometrics for which existing tests are known to exhibit low power. Using a simulation framework common to the literature we show that this approach can improve upon overall accuracy of the traditional unit root test(s) by seventeen percentage points, and the sensitivity by thirty six percentage points.