Context-Emotion Aware Therapeutic Dialogue Generation: A Multi-component Reinforcement Learning Approach to Language Models for Mental Health Support

TL;DR

This paper enhances GPT-2 for therapeutic dialogue generation by integrating multi-component reinforcement learning and emotional context, resulting in more relevant, empathetic, and clinically aligned responses for mental health support.

Contribution

It introduces a novel multi-component reward function and input restructuring to improve emotional and contextual coherence in LLM-based therapeutic dialogues.

Findings

Significant improvements in BLEU, ROUGE, and METEOR scores.

High contextual relevance and professionalism confirmed by LLM evaluation.

Emotion accuracy increased to 99.34% with RL.

Abstract

Mental health disorders impose a substantial global socioeconomic burden. While large language models (LLMs) offer 24/7, non-judgmental interactions to address this gap, pretrained models lack contextual coherence and emotional alignment for appropriate therapeutic dialogue. Existing methods suffer from three critical methodological gaps: 1) Supervised Fine-Tuning (SFT) produces repetitive, context-insensitive outputs that fail to balance clinical accuracy with genuine empathy; 2) Reinforcement Learning (RL)-based therapeutic systems rely on generic reward functions (e.g., BLEU, ROUGE) that prioritise lexical similarity over clinical-specific emotional appropriateness and contextual relevance; 3) LLMs are resource-intensive and pose data privacy risks, making local deployment in clinical settings infeasible. To address these gaps, this study investigates the application of SFT and RL techniques to enhance GPT-2's capacity for therapeutic dialogue generation. The methodology restructured input formats to enable simultaneous processing of contextual information and emotional states alongside user input, employing a novel multi-component reward function that explicitly aligns model outputs with professional therapeutic logic (not just lexical overlap) and annotated emotions. Results demonstrated substantial improvements through RLs over baseline GPT-2 across multiple evaluation metrics: BLEU (0.0111), ROUGE-1 (0.1397), ROUGE-2 (0.0213), ROUGE-L (0.1317), and METEOR (0.0581). LLM evaluation confirmed high contextual relevance and professionalism, while RL achieved 99.34% emotion accuracy compared to 66.96% for baseline GPT-2. These findings demonstrate RL's effectiveness in developing therapeutic dialogue systems that can serve as valuable assistive tools for therapists, while maintaining essential human clinical oversight.