Conformal Depression Prediction

TL;DR

This paper introduces conformal depression prediction (CDP), a method that quantifies uncertainty in depression prediction models using conformal prediction, providing valid confidence intervals without retraining, and proposes an improved version, CDP-ACC, for more adaptive and tighter intervals.

Contribution

The paper presents a novel conformal prediction framework for depression prediction that offers valid uncertainty quantification and introduces CDP-ACC for improved adaptive confidence intervals.

Findings

CDP provides valid confidence intervals with theoretical coverage guarantees.

CDP-ACC achieves tighter, adaptive prediction intervals.

Empirical results show CDP-ACC outperforms baseline methods on AVEC datasets.

Abstract

While existing depression prediction methods based on deep learning show promise, their practical application is hindered by the lack of trustworthiness, as these deep models are often deployed as black box models, leaving us uncertain on the confidence of their predictions. For high-risk clinical applications like depression prediction, uncertainty quantification is essential in decision-making. In this paper, we introduce conformal depression prediction (CDP), a depression prediction method with uncertainty quantification based on conformal prediction (CP), giving valid confidence intervals with theoretical coverage guarantees for the model predictions. CDP is a plug-and-play module that requires neither model retraining nor an assumption about the depression data distribution. As CDP provides only an average coverage guarantee across all inputs rather than per-input performance guarantee, we further propose CDP-ACC, an improved conformal prediction with approximate conditional coverage. CDP-ACC firstly estimates the prediction distribution through neighborhood relaxation, and then introduces a conformal score function by constructing nested sequences, so as to provide a tighter prediction interval adaptive to specific input. We empirically demonstrate the application of CDP in uncertainty-aware facial depression prediction, as well as the effectiveness and superiority of CDP-ACC on the AVEC 2013 and AVEC 2014 datasets. Our code is publicly available at https://github.com/PushineLee/CDP.