When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews

TL;DR

This paper reveals that models for depression detection in clinical interviews often rely on interviewer prompts rather than patient language, highlighting a bias that inflates performance and undermines interpretability.

Contribution

It identifies a systematic bias from interviewer prompts in semi-structured interviews and demonstrates the importance of analyzing decision evidence by speaker to ensure genuine linguistic understanding.

Findings

Models exploit interviewer prompts to achieve high accuracy.

Restricting to participant utterances distributes decision evidence more broadly.

Including interviewer prompts inflates performance by leveraging script artifacts.

Abstract

Automatic depression detection from doctor-patient conversations has gained momentum thanks to the availability of public corpora and advances in language modeling. However, interpretability remains limited: strong performance is often reported without revealing what drives predictions. We analyze three datasets: ANDROIDS, DAIC-WOZ, E-DAIC and identify a systematic bias from interviewer prompts in semi-structured interviews. Models trained on interviewer turns exploit fixed prompts and positions to distinguish depressed from control subjects, often achieving high classification scores without using participant language. Restricting models to participant utterances distributes decision evidence more broadly and reflects genuine linguistic cues. While semi-structured protocols ensure consistency, including interviewer prompts inflates performance by leveraging script artifacts. Our results highlight a cross-dataset, architecture-agnostic bias and emphasize the need for analyses that localize decision evidence by time and speaker to ensure models learn from participants' language.