Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs

TL;DR

This paper develops an LLM-based method to assess and understand the therapeutic alliance in online text counseling, addressing unique challenges and providing insights to improve counselor-client relationships.

Contribution

It introduces a theoretically grounded framework and an automatic LLM-based approach for characterizing therapeutic alliance in online text counseling, with supporting guidelines and evidence extraction.

Findings

Effective LLM-based identification of therapeutic alliance

Challenges faced by counselors in online relationship building

Potential of LLM feedback to improve counseling relationships

Abstract

Robust therapeutic relationships between counselors and clients are fundamental to counseling effectiveness. The assessment of therapeutic alliance is well-established in traditional face-to-face therapy but may not directly translate to text-based settings. With millions of individuals seeking support through online text-based counseling, understanding the relationship in such contexts is crucial. In this paper, we present an automatic approach using large language models (LLMs) to understand the development of therapeutic alliance in text-based counseling. We adapt a theoretically grounded framework specifically to the context of online text-based counseling and develop comprehensive guidelines for characterizing the alliance. We collect a comprehensive counseling dataset and conduct multiple expert evaluations on a subset based on this framework. Our LLM-based approach, combined with guidelines and simultaneous extraction of supportive evidence underlying its predictions, demonstrates effectiveness in identifying the therapeutic alliance. Through further LLM-based evaluations on additional conversations, our findings underscore the challenges counselors face in cultivating strong online relationships with clients. Furthermore, we demonstrate the potential of LLM-based feedback mechanisms to enhance counselors' ability to build relationships, supported by a small-scale proof-of-concept.