OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding

TL;DR

OphNet is a comprehensive, large-scale ophthalmic surgical video dataset with detailed annotations, designed to advance AI-driven surgical workflow understanding and facilitate temporal analysis in diverse ophthalmic procedures.

Contribution

It introduces OphNet, the largest annotated ophthalmic surgical video benchmark, with diverse procedures, hierarchical annotations, and time-localized labels, addressing previous dataset limitations.

Findings

Largest ophthalmic surgical video dataset to date

Enables comprehensive temporal and hierarchical workflow analysis

Supports development of AI systems for surgical understanding

Abstract

Surgical scene perception via videos is critical for advancing robotic surgery, telesurgery, and AI-assisted surgery, particularly in ophthalmology. However, the scarcity of diverse and richly annotated video datasets has hindered the development of intelligent systems for surgical workflow analysis. Existing datasets face challenges such as small scale, lack of diversity in surgery and phase categories, and absence of time-localized annotations. These limitations impede action understanding and model generalization validation in complex and diverse real-world surgical scenarios. To address this gap, we introduce OphNet, a large-scale, expert-annotated video benchmark for ophthalmic surgical workflow understanding. OphNet features: 1) A diverse collection of 2,278 surgical videos spanning 66 types of cataract, glaucoma, and corneal surgeries, with detailed annotations for 102 unique surgical phases and 150 fine-grained operations. 2) Sequential and hierarchical annotations for each surgery, phase, and operation, enabling comprehensive understanding and improved interpretability. 3) Time-localized annotations, facilitating temporal localization and prediction tasks within surgical workflows. With approximately 285 hours of surgical videos, OphNet is about 20 times larger than the largest existing surgical workflow analysis benchmark. Code and dataset are available at: https://minghu0830.github.io/OphNet-benchmark/.