Assessing the educational quality of YouTube videos on celiac plexus blocks: Expert review and AI-based evaluation

TL;DR

This study evaluates the educational quality of YouTube videos on celiac plexus blocks and finds most videos lack reliability, with no significant difference between professional and nonprofessional sources.

Contribution

The study introduces an AI-based evaluation using ChatGPT-4o alongside expert review to assess YouTube videos on a complex medical procedure.

Findings

Only 18% of videos contained highly reliable information based on expert ratings.

65% of videos were classified as useful, but inter-rater reliability between experts was poor to moderate.

AI (ChatGPT-4o) showed poor agreement with human expert assessments.

Abstract

YouTube is an open-access platform increasingly used for both medical and patient education, but its user-generated content is not subject to peer review and shows wide variability in accuracy and quality. Celiac plexus blocks are technically complex procedures that are presented on YouTube, yet the educational quality of these instructional videos has not yet been systematically evaluated.

To evaluate the educational quality of YouTube videos on celiac plexus blocks and to explore the utility of ChatGPT-4o as a secondary, adjunctive tool for assessing video quality.

YouTube was searched on June 2nd, 2025 using the keywords “celiac plexus neurolysis,” “celiac block for cancer pain,” “celiac plexus block,” and “celiac plexus injection.” The 17 most-viewed videos were independently evaluated by two board-certified chronic pain physicians and by ChatGPT-4o using a modified DISCERN scale (mDISCERN), the Global Quality Scale (GQS), and a usefulness classification.

Based on human expert ratings, only 18 % of videos contained highly reliable information as assessed by the mDISCERN scale, and 24 % demonstrated moderate to excellent information quality on the Global Quality Scale. Overall, 65 % of videos were classified as useful. Inter-rater reliability between human experts ranged from poor to moderate across the three scales of evaluation, while agreement between human expert and ChatGPT-4o assessment was poor.

The educational quality of YouTube videos on celiac plexus blocks was generally poor. Unlike similar studies investigating other procedures, the quality of videos produced by physician and hospital sources was not better than that of videos by nonacademic sources. These findings highlight the need to improve the quality of educational content produced by physicians, hospitals, and professional societies.