HieraSurg: Hierarchy-Aware Diffusion Model for Surgical Video Generation

TL;DR

HieraSurg is a hierarchy-aware diffusion framework that generates realistic surgical videos by integrating surgical phases, actions, and segmentation maps, improving consistency and detail over prior methods.

Contribution

The paper introduces a novel two-stage diffusion model that incorporates hierarchical surgical information for more accurate and detailed surgical video synthesis.

Findings

Outperforms prior methods quantitatively and qualitatively

Generates higher frame-rate videos with better detail

Shows strong generalization and adherence to segmentation maps

Abstract

Surgical Video Synthesis has emerged as a promising research direction following the success of diffusion models in general-domain video generation. Although existing approaches achieve high-quality video generation, most are unconditional and fail to maintain consistency with surgical actions and phases, lacking the surgical understanding and fine-grained guidance necessary for factual simulation. We address these challenges by proposing HieraSurg, a hierarchy-aware surgical video generation framework consisting of two specialized diffusion models. Given a surgical phase and an initial frame, HieraSurg first predicts future coarse-grained semantic changes through a segmentation prediction model. The final video is then generated by a second-stage model that augments these temporal segmentation maps with fine-grained visual features, leading to effective texture rendering and integration of semantic information in the video space. Our approach leverages surgical information at multiple levels of abstraction, including surgical phase, action triplets, and panoptic segmentation maps. The experimental results on Cholecystectomy Surgical Video Generation demonstrate that the model significantly outperforms prior work both quantitatively and qualitatively, showing strong generalization capabilities and the ability to generate higher frame-rate videos. The model exhibits particularly fine-grained adherence when provided with existing segmentation maps, suggesting its potential for practical surgical applications.