Strategy-Supervised Autonomous Laparoscopic Camera Control via Event-Driven Graph Mining

TL;DR

This paper introduces a strategy-supervised framework for autonomous laparoscopic camera control that combines event-driven graph mining with vision-language inference, improving stability and safety in surgical views.

Contribution

It presents a novel approach that mines camera-handling strategies from surgical videos and applies them in real-time control with safety constraints and human-in-the-loop capabilities.

Findings

Event parsing achieves F1-score 0.86 for temporal localization.

Mined strategies align well with expert interpretation (cluster purity 0.81).

System reduces camera errors by over 35% and image shaking by over 62% in ex vivo tests.

Abstract

Autonomous laparoscopic camera control must maintain a stable and safe surgical view under rapid tool-tissue interactions while remaining interpretable to surgeons. We present a strategy-grounded framework that couples high-level vision-language inference with low-level closed-loop control. Offline, raw surgical videos are parsed into camera-relevant temporal events (e.g., interaction, working-distance deviation, and view-quality degradation) and structured as attributed event graphs. Mining these graphs yields a compact set of reusable camera-handling strategy primitives, which provide structured supervision for learning. Online, a fine-tuned Vision-Language Model (VLM) processes the live laparoscopic view to predict the dominant strategy and discrete image-based motion commands, executed by an IBVS-RCM controller under strict safety constraints; optional speech input enables intuitive human-in-the-loop conditioning. On a surgeon-annotated dataset, event parsing achieves reliable temporal localization (F1-score 0.86), and the mined strategies show strong semantic alignment with expert interpretation (cluster purity 0.81). Extensive ex vivo experiments on silicone phantoms and porcine tissues demonstrate that the proposed system outperforms junior surgeons in standardized camera-handling evaluations, reducing field-of-view centering error by 35.26% and image shaking by 62.33%, while preserving smooth motion and stable working-distance regulation.