Can LLM-Simulated Practice and Feedback Upskill Human Counselors? A Randomized Study with 90+ Novice Counselors

TL;DR

This study demonstrates that LLM-based simulated practice combined with structured feedback significantly enhances novice counselors' client-centered skills and empathy, outperforming practice alone in a randomized trial with over 90 participants.

Contribution

The paper introduces an LLM-simulated training system with feedback for counselor skill development and provides empirical evidence of its effectiveness through a randomized study.

Findings

Practice with feedback improved client-centered microskills.

Practice alone did not improve and even declined in empathy.

Feedback helped adopt a more client-centered listening approach.

Abstract

The growing demand for accessible mental health support requires training more counselors, yet existing approaches remain resource-intensive and difficult to scale. LLMs can realistically simulate patients and generate actionable feedback for training, but their actual impact on novice counselor skill development remains unknown. We developed an LLM-simulated practice and feedback system and conducted a randomized study with 94 novice counselors, comparing practice alone versus practice with feedback. We evaluated behavioral performance, self-efficacy, and qualitative reflections. Results showed the practice-and-feedback group improved in client-centered microskills (reflections, questions), while the practice-alone group showed no improvements. For empathy, the practice-alone group declined over time and performed significantly worse than the feedback group. Qualitative interviews reinforced these findings: feedback helped participants adopt a client-centered listening approach, while practice-alone participants remained solution-oriented. These results suggest LLM-based training systems can promote effective skill development, and combining simulated practice with structured feedback is critical for meaningful improvement.