Hierarchical Attention Network for Evaluating Therapist Empathy in Counseling Session

TL;DR

This paper introduces a hierarchical attention-based neural network that analyzes acoustic speech features to accurately assess therapist empathy levels in counseling sessions, highlighting the importance of considering multiple speaker turns and entire sessions.

Contribution

The study presents a novel hierarchical recurrent network with two-level attention mechanisms for empathy detection solely from acoustic features, achieving 72.1% accuracy.

Findings

Both therapist and client speech contribute to empathy prediction.

Multiple consecutive turns (2-6) provide useful cues for detecting empathy.

Whole session context influences empathy ratings more than individual turns.

Abstract

Counseling typically takes the form of spoken conversation between a therapist and a client. The empathy level expressed by the therapist is considered to be an essential quality factor of counseling outcome. This paper proposes a hierarchical recurrent network combined with two-level attention mechanisms to determine the therapist's empathy level solely from the acoustic features of conversational speech in a counseling session. The experimental results show that the proposed model can achieve an accuracy of $72.1\%$ in classifying the therapist's empathy level as being ``high" or ``low". It is found that the speech from both the therapist and the client are contributing to predicting the empathy level that is subjectively rated by an expert observer. By analyzing speaker turns assigned with high attention weights, it is observed that $2$ to $6$ consecutive turns should be considered together to provide useful clues for detecting empathy, and the observer tends to take the whole session into consideration when rating the therapist empathy, instead of relying on a few specific speaker turns.