Effectiveness of ChatGPT in explaining complex medical reports to patients

TL;DR

This study evaluates ChatGPT's ability to explain complex medical reports to patients, revealing strengths and limitations in accuracy, language appropriateness, and integration into clinical workflows.

Contribution

It provides an empirical assessment of ChatGPT's effectiveness in translating dense medical reports for patient understanding, highlighting key challenges to clinical adoption.

Findings

ChatGPT can generate explanations but with inaccuracies.

Language may be inappropriate or lack personalization.

Trust and workflow integration remain significant challenges.

Abstract

Electronic health records contain detailed information about the medical condition of patients, but they are difficult for patients to understand even if they have access to them. We explore whether ChatGPT (GPT 4) can help explain multidisciplinary team (MDT) reports to colorectal and prostate cancer patients. These reports are written in dense medical language and assume clinical knowledge, so they are a good test of the ability of ChatGPT to explain complex medical reports to patients. We asked clinicians and lay people (not patients) to review explanations and responses of ChatGPT. We also ran three focus groups (including cancer patients, caregivers, computer scientists, and clinicians) to discuss output of ChatGPT. Our studies highlighted issues with inaccurate information, inappropriate language, limited personalization, AI distrust, and challenges integrating large language models (LLMs) into clinical workflow. These issues will need to be resolved before LLMs can be used to explain complex personal medical information to patients.