TeleOR: Real-time Telemedicine System for Full-Scene Operating Room

TL;DR

TeleOR is a real-time telemedicine system that enables high-fidelity, bandwidth-efficient 3D reconstruction of operating rooms for remote surgical guidance, overcoming obstacles and dynamic scene challenges.

Contribution

The paper introduces innovative methods for obstacle-aware calibration, selective scene reconstruction, and adaptive data transmission in real-time OR telepresence systems.

Findings

TeleOR achieves high-quality 3D scene reconstruction in real-time.

The system effectively handles scene obstructions and bandwidth limitations.

Experiments demonstrate superior performance over existing methods.

Abstract

The advent of telemedicine represents a transformative development in leveraging technology to extend the reach of specialized medical expertise to remote surgeries, a field where the immediacy of expert guidance is paramount. However, the intricate dynamics of Operating Room (OR) scene pose unique challenges for telemedicine, particularly in achieving high-fidelity, real-time scene reconstruction and transmission amidst obstructions and bandwidth limitations. This paper introduces TeleOR, a pioneering system designed to address these challenges through real-time OR scene reconstruction for Tele-intervention. TeleOR distinguishes itself with three innovative approaches: dynamic self-calibration, which leverages inherent scene features for calibration without the need for preset markers, allowing for obstacle avoidance and real-time camera adjustment; selective OR reconstruction, focusing on dynamically changing scene segments to reduce reconstruction complexity; and viewport-adaptive transmission, optimizing data transmission based on real-time client feedback to efficiently deliver high-quality 3D reconstructions within bandwidth constraints. Comprehensive experiments on the 4D-OR surgical scene dataset demostrate the superiority and applicability of TeleOR, illuminating the potential to revolutionize tele-interventions by overcoming the spatial and technical barriers inherent in remote surgical guidance.