Depression Diagnosis Dialogue Simulation: Self-improving Psychiatrist with Tertiary Memory

TL;DR

This paper presents the Agent Mental Clinic, a self-improving dialogue system with a tertiary memory structure that enhances depression diagnosis accuracy through simulated patient-psychiatrist interactions, leveraging LLMs without weight modifications.

Contribution

Introduces a novel self-improving psychiatrist agent with tertiary memory and supervision plugins for improved depression diagnosis in dialogue systems.

Findings

Achieves high accuracy in depression and suicide risk diagnosis.

Effective in real-life scenario datasets with limited labeled data.

Demonstrates potential for domain-specific LLM alignment without weight changes.

Abstract

Mental health issues, particularly depressive disorders, present significant challenges in contemporary society, necessitating the development of effective automated diagnostic methods. This paper introduces the Agent Mental Clinic (AMC), a self-improving conversational agent system designed to enhance depression diagnosis through simulated dialogues between patient and psychiatrist agents. To enhance the dialogue quality and diagnosis accuracy, we design a psychiatrist agent consisting of a tertiary memory structure, a dialogue control and reflect plugin that acts as ``supervisor'' and a memory sampling module, fully leveraging the skills reflected by the psychiatrist agent, achieving great accuracy on depression risk and suicide risk diagnosis via conversation. Experiment results on datasets collected in real-life scenarios demonstrate that the system, simulating the procedure of training psychiatrists, can be a promising optimization method for aligning LLMs with real-life distribution in specific domains without modifying the weights of LLMs, even when only a few representative labeled cases are available.