The Reliability and Quality of Videos as Guidance for Gastrointestinal Endoscopy: Cross-Sectional Study

TL;DR

This study finds that YouTube videos about gastrointestinal endoscopy are generally low quality, and while large language models can summarize them, their summaries need improvement.

Contribution

The study evaluates the quality of YouTube endoscopy videos and explores the potential of large language models to summarize them for patients.

Findings

The quality, reliability, and recommendation scores of gastrointestinal endoscopy videos on YouTube were generally unsatisfactory.

LLM summaries of the videos showed high accuracy and completeness but low readability.

Capsule endoscopy videos had significantly lower quality scores compared to gastroscopy and colonoscopy videos.

Abstract

Gastrointestinal endoscopy represents a useful tool for the diagnosis and treatment of gastrointestinal diseases. Video platforms for spreading endoscopy-related knowledge may help patients understand the pros and cons of endoscopy on the premise of ensuring accuracy. However, videos with misinformation may lead to adverse consequences.

This study aims to evaluate the quality of gastrointestinal endoscopy-related videos on YouTube and to assess whether large language models (LLMs) can help patients obtain information from videos more efficiently.

We collected information from YouTube videos about 3 commonly used gastrointestinal endoscopes (gastroscopy, colonoscopy, and capsule endoscopy) and assessed their quality (rated by the modified DISCERN Tool, mDISCERN), reliability (rated by the Journal of the American Medical Association), and recommendation (rated by the Global Quality Score). We tasked LLM with summarizing the video content and assessed it from 3 perspectives: accuracy, completeness, and readability.

A total of 167 videos were included. According to the indicated scoring, the quality, reliability, and recommendation of the 3 gastrointestinal endoscopy-related videos on YouTube were overall unsatisfactory, and the quality of the videos released by patients was particularly poor. Capsule endoscopy yielded a significantly lower Global Quality Score than did gastroscopy and colonoscopy. LLM-based summaries yielded accuracy scores of 4 (IQR 4-5), completeness scores of 4 (IQR 4-5), and readability scores of 2 (IQR 1-2).

The quality of gastrointestinal endoscope-related videos currently on YouTube is poor. Moreover, additional regulatory and improvement strategies are needed in the future. LLM may be helpful in generalizing video-related information, but there is still room for improvement in its ability.

Figures