TopoOR: A Unified Topological Scene Representation for the Operating Room

TL;DR

TopoOR introduces a higher-order topological scene representation for surgical operating rooms, capturing complex multimodal relationships more effectively than traditional graph-based methods, leading to improved safety-critical reasoning.

Contribution

It proposes a novel topological modeling framework and attention mechanism that preserve manifold structure and multimodality in surgical scenes, surpassing existing paradigms.

Findings

Outperforms traditional graph-based methods in safety-critical tasks

Effectively models complex multimodal interactions in the OR

Enhances reasoning accuracy in surgical scene understanding

Abstract

Surgical Scene Graphs abstract the complexity of surgical operating rooms (OR) into a structure of entities and their relations, but existing paradigms suffer from strictly dyadic structural limitations. Frameworks that predominantly rely on pairwise message passing or tokenized sequences flatten the manifold geometry inherent to relational structures and lose structure in the process. We introduce TopoOR, a new paradigm that models multimodal operating rooms as a higher-order structure, innately preserving pairwise and group relationships. By lifting interactions between entities into higher-order topological cells, TopoOR natively models complex dynamics and multimodality present in the OR. This topological representation subsumes traditional scene graphs, thereby offering strictly greater expressivity. We also propose a higher-order attention mechanism that explicitly preserves manifold structure and modality-specific features throughout hierarchical relational attention. In this way, we circumvent combining 3D geometry, audio, and robot kinematics into a single joint latent representation, preserving the precise multimodal structure required for safety-critical reasoning, unlike existing methods. Extensive experiments demonstrate that our approach outperforms traditional graph and LLM-based baselines across sterility breach detection, robot phase prediction, and next-action anticipation