AI chatbots versus human healthcare professionals: a systematic review and meta-analysis of empathy in patient care

TL;DR

This systematic review and meta-analysis found that AI chatbots, especially those using large language models like ChatGPT, are often rated as more empathetic than human healthcare professionals in text-based assessments, highlighting potential for AI in patient care.

Contribution

The study synthesizes existing research comparing AI chatbots and human providers on empathy, providing quantitative evidence that AI can outperform humans in text-based empathy ratings.

Findings

AI chatbots show higher empathy ratings than human providers in most studies.

Meta-analysis indicates a large effect size favoring AI empathy (SMD 0.87).

Most assessments rely on text-based proxy ratings, not direct patient feedback.

Abstract

Background: Empathy is widely recognized for improving patient outcomes, including reduced pain and anxiety and improved satisfaction, and its absence can cause harm. Meanwhile, use of artificial intelligence (AI)-based chatbots in healthcare is rapidly expanding, with one in five general practitioners using generative AI to assist with tasks such as writing letters. Some studies suggest AI chatbots can outperform human healthcare professionals (HCPs) in empathy, though findings are mixed and lack synthesis. Sources of data: We searched multiple databases for studies comparing AI chatbots using large language models with human HCPs on empathy measures. We assessed risk of bias with ROBINS-I and synthesized findings using random-effects meta-analysis where feasible, whilst avoiding double counting. Areas of agreement: We identified 15 studies (2023-2024). Thirteen studies reported statistically significantly higher empathy ratings for AI, with only two studies situated in dermatology favouring human responses. Of the 15 studies, 13 provided extractable data and were suitable for pooling. Meta-analysis of those 13 studies, all utilising ChatGPT-3.5/4, showed a standardized mean difference of 0.87 (95% CI, 0.54-1.20) favouring AI (P < .00001), roughly equivalent to a two-point increase on a 10-point scale. Areas of controversy: Studies relied on text-based assessments that overlook non-verbal cues and evaluated empathy through proxy raters. Growing points: Our findings indicate that, in text-only scenarios, AI chatbots are frequently perceived as more empathic than human HCPs. Areas timely for developing research: Future research should validate these findings with direct patient evaluations and assess whether emerging voice-enabled AI systems can deliver similar empathic advantages.