You Never Know a Person, You Only Know Their Defenses: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations

TL;DR

This paper presents PsyDefConv, a new dialogue corpus and a four-stage pipeline for detecting psychological defense mechanisms in supportive conversations, aiding mental health research and clinical assessment.

Contribution

It introduces a novel annotated corpus and an evidence-based annotation pipeline for defense mechanisms, along with benchmark results using language models.

Findings

The corpus contains 200 dialogues with defense annotations and good inter-annotator agreement.

The DMRS Co-Pilot reduces annotation time by 22.4%.

Language models achieve around 30% macro F1-score, indicating room for improvement.

Abstract

Psychological defenses are strategies, often automatic, that people use to manage distress. Rigid or overuse of defenses is negatively linked to mental health and shapes what speakers disclose and how they accept or resist help. However, defenses are complex and difficult to reliably measure, particularly in clinical dialogues. We introduce PsyDefConv, a dialogue corpus with help seeker utterances labeled for defense level, and DMRS Co-Pilot, a four-stage pipeline that provides evidence-based pre-annotations. The corpus contains 200 dialogues and 4709 utterances, including 2336 help seeker turns, with labeling and Cohen's kappa 0.639. In a counterbalanced study, the co-pilot reduced average annotation time by 22.4%. In expert review, it averaged 4.62 for evidence, 4.44 for clinical plausibility, and 4.40 for insight on a seven-point scale. Benchmarks with strong language models in zero-shot and fine-tuning settings demonstrate clear headroom, with the best macro F1-score around 30% and a tendency to overpredict mature defenses. Corpus analyses confirm that mature defenses are most common and reveal emotion-specific deviations. We will release the corpus, annotations, code, and prompts to support research on defensive functioning in language.