# Exploring Patient Perspectives, Engagement, and Output Quality in Doctor-Supervised Use of Artificial Intelligence During Informed Consent Consultation With ChatGPT and Retrieval Augmented Generation (RAG): Quantitative Exploratory Study

## TL;DR

This study explores how AI tools like ChatGPT and RAG affect patient satisfaction and understanding during preoperative consent for hip surgery, finding that RAG improves AI performance but patients still prefer human doctors.

## Contribution

The study introduces a customized ChatGPT model enhanced with RAG for preoperative education and evaluates its impact on patient engagement and information quality.

## Key findings

- ChatGPT+RAG significantly improved patient satisfaction and perceived informedness compared to native ChatGPT.
- The ChatGPT+RAG model had a lower hallucination rate and better response quality than native ChatGPT.
- Patients preferred physician-led consultations even when AI provided more accurate information.

## Abstract

Comprehensive preoperative education is essential for optimizing outcomes and ensuring informed consent in patients undergoing total hip arthroplasty (THA). Emerging artificial intelligence (AI) tools, such as ChatGPT, offer scalable support for patient education, but their clinical application requires rigorous evaluation to ensure accuracy, safety, and trust.

This study assessed patients’ preferences and satisfaction with AI-assisted informed consent in THA, comparing traditional physician consultations to those supported by native ChatGPT and a customized version enhanced with retrieval-augmented generation (RAG). It also examined how state anxiety and general attitudes toward AI affect preferences for AI-supported consent and whether RAG integration improves ChatGPT response quality.

A total of 36 patients scheduled for elective THA were assigned to one of three groups (12 each): (1) standard physician-only consultations (control), (2) physician-assisted consultations supported by native ChatGPT, and (3) supported by ChatGPT enhanced through RAG. Data collection involved standardized Likert scale questionnaires assessing patient satisfaction with the consent process, perceived informedness, anxiety levels, and attitudes toward AI. The ChatGPT responses were independently evaluated by physicians for relevance, accuracy, clarity, completeness, adherence to evidence-based guidelines, and appropriate length. Instances of hallucinations, factually incorrect or misleading outputs, were identified and rated by severity. Statistical analyses compared outcomes across groups and explored associations.

Patients interacting with the ChatGPT+RAG model reported significantly higher satisfaction levels with information delivery (P=.01) and perceived level of informedness (P=.01) than those using the native ChatGPT model. The mean number of patient questions in the control group was 20, compared with 39 in the native ChatGPT group (P=.06) and 52 in the ChatGPT+RAG group (P=.002). The majority of participants across all groups preferred a human clinician providing less accurate information over a more accurate AI-only assistant. These preferences were not influenced by sociodemographic variables (age, gender, and education), health literacy, state anxiety, or general attitudes toward AI. The ChatGPT+RAG model outperformed the native ChatGPT model across all evaluated response quality dimensions (all P<.01) and exhibited a significantly lower hallucination rate (5/52, 10% versus 15/39, 38%; P=.002).

Integrating RAG with ChatGPT significantly improves the quality, clarity, and reliability of preoperative information, enhancing patient satisfaction and engagement beyond native ChatGPT. However, patients maintain a strong preference for physician-led informed consent, underscoring the role of AI chatbots as complementary tools rather than replacements. These findings support the cautious adoption of customized AI assistants to augment, not substitute, human interaction in surgical consent processes.