Automatic Detection and Classification of Cognitive Distortions in Mental Health Text

TL;DR

This paper introduces a machine learning framework for automatically detecting and classifying 15 cognitive distortions in mental health texts, aiding in mental health assessment and therapy. It demonstrates high accuracy in distinguishing distorted from non-distorted passages and provides insights into thematic elements of distortions.

Contribution

The study presents a novel machine learning approach for classifying cognitive distortions in free text, using two new datasets, and explores thematic relationships between distortions for mental health treatment.

Findings

Weighted F1 score of 0.88 for detecting distorted vs. non-distorted passages.

F1 scores of 0.68 and 0.45 for classifying 15 distortion types in two datasets.

Identification of discriminative words and phrases to improve mental health interventions.

Abstract

In cognitive psychology, automatic and self-reinforcing irrational thought patterns are known as cognitive distortions. Left unchecked, patients exhibiting these types of thoughts can become stuck in negative feedback loops of unhealthy thinking, leading to inaccurate perceptions of reality commonly associated with anxiety and depression. In this paper, we present a machine learning framework for the automatic detection and classification of 15 common cognitive distortions in two novel mental health free text datasets collected from both crowdsourcing and a real-world online therapy program. When differentiating between distorted and non-distorted passages, our model achieved a weighted F1 score of 0.88. For classifying distorted passages into one of 15 distortion categories, our model yielded weighted F1 scores of 0.68 in the larger crowdsourced dataset and 0.45 in the smaller online counseling dataset, both of which outperformed random baseline metrics by a large margin. For both tasks, we also identified the most discriminative words and phrases between classes to highlight common thematic elements for improving targeted and therapist-guided mental health treatment. Furthermore, we performed an exploratory analysis using unsupervised content-based clustering and topic modeling algorithms as first efforts towards a data-driven perspective on the thematic relationship between similar cognitive distortions traditionally deemed unique. Finally, we highlight the difficulties in applying mental health-based machine learning in a real-world setting and comment on the implications and benefits of our framework for improving automated delivery of therapeutic treatment in conjunction with traditional cognitive-behavioral therapy.