Quantifying improvement of psychotic symptoms in clozapine-treated schizophrenia: clinical note analysis with large language models

TL;DR

This study uses large language models to analyze clinical notes and track improvements in schizophrenia symptoms during clozapine treatment.

Contribution

The novel use of LLMs to quantify treatment-related symptom improvement in schizophrenia from unstructured clinical notes.

Findings

LLM-based ratings showed significant decreases in symptoms like anxiety and hallucinatory behavior during clozapine treatment.

POS and LIWC analyses revealed increased use of adjectives and more positive emotional expressions later in treatment.

LLMs can extract meaningful clinical symptom data from unstructured EHR notes, capturing treatment-related changes in psychosis.

Abstract

Symptoms of schizophrenia are often reflected in patients’ speech. Natural language processing (NLP) approaches enable quantitative assessment of language-related symptoms in schizophrenia. Previous applications have primarily focused on acute psychopathology or predicting the onset or relapse of psychosis rather than treatment-related improvements. Although electronic health records (EHRs) contain rich longitudinal data, unstructured notes hinder structured quantifications. We applied recent large language models (LLMs) to evaluate symptoms based on speech content recorded in EHRs. We analyzed 5,275 clinical notes from 30 patients with treatment-resistant schizophrenia undergoing clozapine treatment. Three state-of-the-art LLMs rated according to the Brief Psychiatric Rating Scale (BPRS). Complementary analysis included parts-of-speech (POS), bag-of-words (BoW), bigram and Linguistic Inquiry and Word Count (LIWC) analyses. LLM-based BPRS ratings revealed significant decreases in Anxiety, Conceptual Disorganization, Suspiciousness, Unusual Thought Content, Hallucinatory behavior, and Depressive Mood during clozapine treatment. POS analysis indicated an increased use of adjectives per sentence, while LIWC analysis revealed more positive emotional expressions during the later phase of treatment. These findings demonstrate that LLMs can extract clinically meaningful symptom information from unstructured clinical text and capture treatment-related changes in psychosis. This approach premises a low-burden method for supporting clinical judgment using routinely collected EHR data.

The online version contains supplementary material available at 10.1038/s41598-026-39676-0.

Figures