Development of an evaluation framework for robotic total mesorectal excision videos: a review and comparison of medical professional and public video resources

TL;DR

This paper evaluates the quality of online surgical videos for robotic total mesorectal excision using a new checklist, finding significant differences between platforms like WebSurg, C-SATS, and YouTube.

Contribution

The study introduces a novel quality assessment checklist for robotic TME videos and compares educational content across different platforms.

Findings

WebSurg and YouTube videos scored significantly higher in quality than C-SATS videos.

YouTube videos showed higher usability scores compared to WebSurg and C-SATS.

The overall quality of online robotic TME videos was found to be heterogeneous.

Abstract

This study aims to assess the quality of educational surgical videos for robotic total mesorectal excision (TME), across widely used open-source platforms, using a newly designed quality assessment checklist.

The checklist was developed by using existing society guidelines, such as the European Academy of Robotic Colorectal Surgery, comprising four key sections: (i) usability of the platform, (ii) video component, (iii) intraoperative techniques and (iv) other information (including case presentation and outcomes). Videos were identified using the search terms ‘Robotic TME’ from surgical education platforms (WebSurg, C-SATS and Touch Surgery) and YouTube, between January 2016 and July 2024. All videos displaying robotic TME were reviewed and scored using the quality assessment tool (/12), and the videos across the platforms were subsequently compared.

A total of 113 videos were scored using the checklist: 63 surgical education platform (10 WebSurg and 53 C-SATS) and 50 YouTube videos. The total median checklist score achieved by WebSurg (9 [IQR 8–9] and YouTube videos (8 [IQR 7–10]) was significantly higher than CSAT-S videos (4 [IQR 4–5]; p < 0.001). The usability of platform scores for YouTube was significantly higher than WebSurg and C-SATS videos (p < 0.001). Scores for video components, intraoperative techniques and other information were higher across WebSurg and YouTube videos when compared to C-SATS (p < 0.001); however, there was no significant difference between WebSurg and YouTube for each domain.

The overall educational quality of online robotic TME videos was found to be generally heterogeneous, with WebSurg and YouTube videos demonstrating higher scores based on the checklist. A new quality assessment tool has been proposed for robotic TME videos, which has the potential to improve the reliability and value of published video research.

The online version contains supplementary material available at 10.1007/s00384-025-04914-w.

Figures