Deep-testing: the case of dependence detection

TL;DR

This paper introduces deep-testing, a novel hypothesis testing method using deep neural networks, demonstrating superior power in independence testing compared to existing methods.

Contribution

The paper proposes deep-testing, a new approach that leverages deep learning for hypothesis testing, especially effective in complex dependence structures.

Findings

Deep-testing achieves the highest overall power in independence testing.

It outperforms nineteen competing methods across various complex dependence structures.

The approach confirms the viability of deep learning in classical statistical inference.

Abstract

Deep learning methods have proved highly effective for classification and image recognition problems. In this paper, we ask whether this success can be transferred to hypothesis testing: if a neural network can distinguish, for example, an image of a handwritten digit from another, can it also distinguish an "image of a sample" (such as a scatter plot) generated under a given statistical model from one generated outside that model? Motivated by this idea, we propose a novel procedure called deep-testing, which approaches the classical inferential problem of hypothesis testing through deep learning. More specifically, the test statistic is a classification map learned by a deep neural network from simulated data satisfying the null and alternative hypotheses, leveraging its strong discriminating power to construct a highly powerful test. As a proof of concept, we apply deep-testing to the problem of independence testing, arguably one of the most important problems in statistics. In a large-scale simulation study, deep-testing achieves the highest overall power against nineteen competing methods across a broad range of complex dependence structures, confirming the viability of the proposed approach.