InsightFlow: LLM-Driven Synthesis of Patient Narratives for Mental Health into Causal Models

TL;DR

InsightFlow leverages large language models to automatically generate clinically meaningful causal models from psychotherapy dialogues, aligning with expert practices and aiding mental health case formulation.

Contribution

This work introduces InsightFlow, the first LLM-based system for automatic causal graph generation from therapy transcripts, evaluated against expert annotations.

Findings

Generated graphs have structural similarity comparable to inter-annotator agreement.

High semantic alignment with human-generated graphs.

Expert ratings indicate the graphs are moderately complete, consistent, and useful.

Abstract

Clinical case formulation organizes patient symptoms and psychosocial factors into causal models, often using the 5P framework. However, constructing such graphs from therapy transcripts is time consuming and varies across clinicians. We present InsightFlow, an LLM based approach that automatically generates 5P aligned causal graphs from patient-therapist dialogues. Using 46 psychotherapy intake transcripts annotated by clinical experts, we evaluate LLM generated graphs against human formulations using structural (NetSimile), semantic (embedding similarity), and expert rated clinical criteria. The generated graphs show structural similarity comparable to inter annotator agreement and high semantic alignment with human graphs. Expert evaluations rate the outputs as moderately complete, consistent, and clinically useful. While LLM graphs tend to form more interconnected structures compared to the chain like patterns of human graphs, overall complexity and content coverage are similar. These results suggest that LLMs can produce clinically meaningful case formulation graphs within the natural variability of expert practice. InsightFlow highlights the potential of automated causal modeling to augment clinical workflows, with future work needed to improve temporal reasoning and reduce redundancy.