Cross-Demographic Portability of Deep NLP-Based Depression Models

TL;DR

This study evaluates the generalization of NLP-based depression detection models across different age groups, showing promising portability despite demographic differences, with only modest performance degradation.

Contribution

It demonstrates that depression models trained on younger speakers can effectively generalize to older populations with minimal performance loss.

Findings

Model trained on young speakers achieved AUC=0.82 on same-age test data.

Model tested on senior data achieved AUC=0.76, showing modest degradation.

A subset of seniors with stable health states had AUC=0.81.

Abstract

Deep learning models are rapidly gaining interest for real-world applications in behavioral health. An important gap in current literature is how well such models generalize over different populations. We study Natural Language Processing (NLP) based models to explore portability over two different corpora highly mismatched in age. The first and larger corpus contains younger speakers. It is used to train an NLP model to predict depression. When testing on unseen speakers from the same age distribution, this model performs at AUC=0.82. We then test this model on the second corpus, which comprises seniors from a retirement community. Despite the large demographic differences in the two corpora, we saw only modest degradation in performance for the senior-corpus data, achieving AUC=0.76. Interestingly, in the senior population, we find AUC=0.81 for the subset of patients whose health state is consistent over time. Implications for demographic portability of speech-based applications are discussed.