Stabilizing Temporal Inference Dynamics for Online Surgical Phase Recognition

TL;DR

This paper introduces a unified framework with novel loss and inference components to improve the temporal stability of online surgical phase recognition models, reducing prediction fragmentation and increasing reliability.

Contribution

It proposes a model-agnostic, plug-and-play framework that stabilizes temporal inference dynamics using TEC loss and EGTP, and introduces the TFI metric for evaluation.

Findings

Significant reduction in temporal fragmentation across datasets.

Improved stability without sacrificing frame-wise accuracy.

Framework is effective across multiple backbone models.

Abstract

Online Surgical Phase Recognition (SPR) models can reach high frame-wise accuracy, yet their predictions often lack temporal stability, fragmenting workflow understanding and reducing the reliability of downstream assistance. We show that this instability is not random noise but arises from two mechanisms: early misclassifications corrupt temporal feature states and propagate forward to form error cascades, and phase transitions follow evidence-accumulation dynamics whereas most online SPR systems rely on memoryless frame-wise decisions, making them sensitive to transient confidence fluctuations. We propose a unified Train-Inference-Evaluation framework that explicitly stabilizes temporal inference dynamics using model-agnostic, plug-and-play components. For training, the Temporal Error-Cascade (TEC) loss suppresses error onset and mitigates forward error propagation by stabilizing temporal feature evolution. For inference, the Evidence-Gated Transition Predictor (EGTP) enforces evidence-driven state transitions, allowing phase changes only when accumulated evidence exceeds a confidence boundary. For evaluation, we introduce the Temporal Fragmentation Index (TFI), a reliability-aware metric that quantifies instability-induced temporal disagreement beyond conventional frame-wise and token-based measures. Experiments on Cholec80 and AutoLaparo across three representative backbones show that the proposed framework substantially improves temporal stability and reduces prediction fragmentation, while maintaining or modestly improving frame-wise performance.