Trustworthy AI Psychotherapy: Multi-Agent LLM Workflow for Counseling and Explainable Mental Disorder Diagnosis

TL;DR

This paper introduces DSM5AgentFlow, an LLM-based workflow that autonomously generates diagnostic questionnaires, simulates therapist-client dialogues, and provides explainable mental disorder predictions to enhance trustworthiness in AI mental health tools.

Contribution

It presents the first LLM-based agent workflow for DSM-5 diagnosis, addressing current limitations by enabling autonomous, explainable, and realistic mental health assessments.

Findings

High diagnostic accuracy across tested LLMs

Enhanced conversational realism in simulated dialogues

Improved explainability and transparency of predictions

Abstract

LLM-based agents have emerged as transformative tools capable of executing complex tasks through iterative planning and action, achieving significant advancements in understanding and addressing user needs. Yet, their effectiveness remains limited in specialized domains such as mental health diagnosis, where they underperform compared to general applications. Current approaches to integrating diagnostic capabilities into LLMs rely on scarce, highly sensitive mental health datasets, which are challenging to acquire. These methods also fail to emulate clinicians' proactive inquiry skills, lack multi-turn conversational comprehension, and struggle to align outputs with expert clinical reasoning. To address these gaps, we propose DSM5AgentFlow, the first LLM-based agent workflow designed to autonomously generate DSM-5 Level-1 diagnostic questionnaires. By simulating therapist-client dialogues with specific client profiles, the framework delivers transparent, step-by-step disorder predictions, producing explainable and trustworthy results. This workflow serves as a complementary tool for mental health diagnosis, ensuring adherence to ethical and legal standards. Through comprehensive experiments, we evaluate leading LLMs across three critical dimensions: conversational realism, diagnostic accuracy, and explainability. Our datasets and implementations are fully open-sourced.