Impact of GPT-4–Generated Discharge Letters on Patients’ Medical Comprehension: Prospective Crossover Study

TL;DR

GPT-4 generated discharge letters improved patients' understanding of medical information compared to standard letters, especially for medication and organization details.

Contribution

This study provides empirical evidence that AI-generated patient-centered discharge letters enhance medical comprehension and reduce cognitive load.

Findings

GPT-4 letters improved comprehension of safety-relevant information compared to standard discharge letters.

The greatest improvements were observed in medication and organizational content fields.

Higher-order understanding, such as risk prevention, was less effectively supported by AI-generated letters.

Abstract

Patients often struggle to understand standard hospital discharge letters, increasing the risk of medication errors and misunderstandings. According to cognitive load theory (CLT), complex, information-dense texts can overload working memory and impair comprehension. Artificial intelligence tools that generate patient-centered versions could help reduce extraneous cognitive load and bridge this gap. However, evidence for their effectiveness remains limited.

This study aimed to evaluate whether GPT-4 (OpenAI)–generated patient-centered letters improve standardized patients’ retention and understanding of safety-relevant medical information compared with standard hospital discharge letters, and to explore potential effects on cognitive load as described by CLT.

In this prospective, randomized, crossover study, 48 trained standardized patients received a conventional discharge letter for an assigned disease (out of 3) and its matching GPT-4–generated patient-centered letter. Participants read one version first, identified predefined safety-relevant “learning objectives,” and then repeated the task with the alternate version. The primary outcome was the proportion of learning objectives fully, partially, or not reported. In a secondary analysis, results were stratified by content field (Medication, Organization, Prevention of Complications, Lifestyle/Disease Management) and Bloom taxonomy level (“Remember,” “Understand”).

The letter type significantly influenced comprehension (odds ratio [OR] 1.74, 95% CI 1.45-2.08; P<.001). Patient letters, compared with discharge letters, led to higher rates of fully (490/1073, 45.7% vs 413/1073, 38.5%) or partially (322/1073, 30% vs 287/1073, 26.7%) stated learning objectives and fewer omissions (261/1073, 24.3% vs 373/1073, 34.8%). Participants performed better on “Remember” than on “Understand” learning objectives, regardless of letter type (OR 3.33, 95% CI 1.96-5.88; P<.001). Compared with standard hospital discharge letters, patient letters consistently improved results at both cognitive levels (“Remember”: 278/545, 51% vs 242/545, 44.4%; “Understand”: 212/528, 40.2% vs 171/528, 32.4% fully stated). The effect of patient letters varied by content field (P<.001). The greatest improvements were observed for “Medication” (170/254, 66.9% vs 129/254, 50.8% fully stated) and “Organization” (78/158, 49.4% vs 62/158, 39.2% fully stated). Improvements in the content field “Prevention of Complications” were modest, and those for “Lifestyle/Disease Management” were even smaller across all conditions. A total of 24.3% (261/1073) of key information remained unrecognized.

In this explanatory study, GPT-4–generated patient letters improved comprehension of safety-relevant discharge information among standardized patients, particularly regarding medication and organizational aspects. However, they were less effective in supporting higher-order understanding, such as risk prevention or lifestyle management. These hypothesis-driven findings can be interpreted within a CLT framework and may motivate prospective evaluation of multimodal, iterative supports.

Figures