SurgBox: Agent-Driven Operating Room Sandbox with Surgery Copilot

TL;DR

SurgBox is an agent-driven surgical training and operational support framework that uses large language models and a novel memory mechanism to enhance surgeons' cognitive skills and decision-making in complex neurosurgical procedures.

Contribution

We introduce SurgBox, a novel sandbox framework utilizing LLMs and a new memory mechanism to improve surgical training and real-time decision support.

Findings

Enhanced surgical cognitive capabilities demonstrated in experiments.

Effective support for clinical decision-making validated with real neurosurgical data.

Reduced cognitive workload for surgical teams during procedures.

Abstract

Surgical interventions, particularly in neurology, represent complex and high-stakes scenarios that impose substantial cognitive burdens on surgical teams. Although deliberate education and practice can enhance cognitive capabilities, surgical training opportunities remain limited due to patient safety concerns. To address these cognitive challenges in surgical training and operation, we propose SurgBox, an agent-driven sandbox framework to systematically enhance the cognitive capabilities of surgeons in immersive surgical simulations. Specifically, our SurgBox leverages large language models (LLMs) with tailored Retrieval-Augmented Generation (RAG) to authentically replicate various surgical roles, enabling realistic training environments for deliberate practice. In particular, we devise Surgery Copilot, an AI-driven assistant to actively coordinate the surgical information stream and support clinical decision-making, thereby diminishing the cognitive workload of surgical teams during surgery. By incorporating a novel Long-Short Memory mechanism, our Surgery Copilot can effectively balance immediate procedural assistance with comprehensive surgical knowledge. Extensive experiments using real neurosurgical procedure records validate our SurgBox framework in both enhancing surgical cognitive capabilities and supporting clinical decision-making. By providing an integrated solution for training and operational support to address cognitive challenges, our SurgBox framework advances surgical education and practice, potentially transforming surgical outcomes and healthcare quality. The code is available at https://github.com/franciszchen/SurgBox.