AI and narrative embeddings detect PTSD following childbirth via birth stories

TL;DR

This study shows that AI models can detect postpartum PTSD by analyzing birth stories, offering a new screening method for a common mental health issue.

Contribution

The study introduces a novel ML model using text embeddings that outperforms existing AI tools in detecting childbirth-related PTSD.

Findings

The ML model using ADA embeddings achieved an F1 score of 0.81 in detecting CB-PTSD.

The model outperformed ChatGPT and six other large text-embedding models trained on mental health data.

The approach could be generalized to assess other mental health disorders.

Abstract

Free-text analysis using machine learning (ML)-based natural language processing (NLP) shows promise for diagnosing psychiatric conditions. Chat Generative Pre-trained Transformer (ChatGPT) has demonstrated preliminary initial feasibility for this purpose; however, whether it can accurately assess mental illness remains to be determined. This study evaluates the effectiveness of ChatGPT and the text-embedding-ada-002 (ADA) model in detecting post-traumatic stress disorder following childbirth (CB-PTSD), a maternal postpartum mental illness affecting millions of women annually, with no standard screening protocol. Using a sample of 1295 women who gave birth in the last six months and were 18+ years old, recruited through hospital announcements, social media, and professional organizations, we explore ChatGPT’s and ADA’s potential to screen for CB-PTSD by analyzing maternal childbirth narratives. The PTSD Checklist for DSM-5 (PCL-5; cutoff 31) was used to assess CB-PTSD. By developing an ML model that utilizes numerical vector representation of the ADA model, we identify CB-PTSD via narrative classification. Our model outperformed (F1 score: 0.81) ChatGPT and six previously published large text-embedding models trained on mental health or clinical domains data, suggesting that the ADA model can be harnessed to identify CB-PTSD. Our modeling approach could be generalized to assess other mental health disorders.

Figures