Bridging the Gap between Deep Learning and Hypothesis-Driven Analysis via Permutation Testing

TL;DR

This paper introduces a permutation testing method that integrates hypothesis testing with deep learning to identify statistically significant factors in neuropsychological data, demonstrated on adolescent depression risk factors.

Contribution

It presents a scalable, flexible approach combining permutation testing with deep learning for hypothesis-driven analysis in neuroscience research.

Findings

Successfully identified risk factor categories for depression

Linked deep learning findings to statistical significance

Applied to adolescent neuropsychological assessments

Abstract

A fundamental approach in neuroscience research is to test hypotheses based on neuropsychological and behavioral measures, i.e., whether certain factors (e.g., related to life events) are associated with an outcome (e.g., depression). In recent years, deep learning has become a potential alternative approach for conducting such analyses by predicting an outcome from a collection of factors and identifying the most "informative" ones driving the prediction. However, this approach has had limited impact as its findings are not linked to statistical significance of factors supporting hypotheses. In this article, we proposed a flexible and scalable approach based on the concept of permutation testing that integrates hypothesis testing into the data-driven deep learning analysis. We apply our approach to the yearly self-reported assessments of 621 adolescent participants of the National Consortium of Alcohol and Neurodevelopment in Adolescence (NCANDA) to predict negative valence, a symptom of major depressive disorder according to the NIMH Research Domain Criteria (RDoC). Our method successfully identifies categories of risk factors that further explain the symptom.