MentalSeek-Dx: Towards Progressive Hypothetico-Deductive Reasoning for Real-world Psychiatric Diagnosis

TL;DR

This paper introduces MentalSeek-Dx, a specialized language model trained to emulate clinical reasoning for psychiatric diagnosis, and presents MentalDx Bench, a new real-world diagnostic benchmark with 712 annotated records.

Contribution

It develops MentalSeek-Dx, a novel LLM that internalizes clinical reasoning, and creates MentalDx Bench, the first benchmark for disorder-level psychiatric diagnosis in real-world settings.

Findings

MentalSeek-Dx outperforms existing models on MentalDx Bench.

Current LLMs excel at broad categories but fail at detailed diagnoses.

MentalSeek-Dx achieves state-of-the-art performance with only 14B parameters.

Abstract

Mental health disorders represent a burgeoning global public health challenge. While Large Language Models (LLMs) have demonstrated potential in psychiatric assessment, their clinical utility is severely constrained by benchmarks that lack ecological validity and fine-grained diagnostic supervision. To bridge this gap, we introduce \textbf{MentalDx Bench}, the first benchmark dedicated to disorder-level psychiatric diagnosis within real-world clinical settings. Comprising 712 de-identified electronic health records annotated by board-certified psychiatrists under ICD-11 guidelines, the benchmark covers 76 disorders across 16 diagnostic categories. Evaluation of 18 LLMs reveals a critical \textit{paradigm misalignment}: strong performance at coarse diagnostic categorization contrasts with systematic failure at disorder-level diagnosis, underscoring a gap between pattern-based modeling and clinical hypothetico-deductive reasoning. In response, we propose \textbf{MentalSeek-Dx}, a medical-specialized LLM trained to internalize this clinical reasoning process through supervised trajectory construction and curriculum-based reinforcement learning. Experiments on MentalDx Bench demonstrate that MentalSeek-Dx achieves state-of-the-art (SOTA) performance with only 14B parameters, establishing a clinically grounded framework for reliable psychiatric diagnosis.