Benchmarking Robustness of Endoscopic Depth Estimation with Synthetically Corrupted Data

TL;DR

This paper introduces a benchmark and a new robustness metric for endoscopic depth estimation models, emphasizing the importance of resilience to image corruptions for surgical safety and precision.

Contribution

It presents a comprehensive dataset with synthetic corruptions, a novel robustness score (DERS), and an analysis framework to evaluate and improve model reliability in surgical settings.

Findings

Models show decreased accuracy under corruptions

DERS effectively measures robustness and error

Benchmark encourages development of more resilient models

Abstract

Accurate depth perception is crucial for patient outcomes in endoscopic surgery, yet it is compromised by image distortions common in surgical settings. To tackle this issue, our study presents a benchmark for assessing the robustness of endoscopic depth estimation models. We have compiled a comprehensive dataset that reflects real-world conditions, incorporating a range of synthetically induced corruptions at varying severity levels. To further this effort, we introduce the Depth Estimation Robustness Score (DERS), a novel metric that combines measures of error, accuracy, and robustness to meet the multifaceted requirements of surgical applications. This metric acts as a foundational element for evaluating performance, establishing a new paradigm for the comparative analysis of depth estimation technologies. Additionally, we set forth a benchmark focused on robustness for the evaluation of depth estimation in endoscopic surgery, with the aim of driving progress in model refinement. A thorough analysis of two monocular depth estimation models using our framework reveals crucial information about their reliability under adverse conditions. Our results emphasize the essential need for algorithms that can tolerate data corruption, thereby advancing discussions on improving model robustness. The impact of this research transcends theoretical frameworks, providing concrete gains in surgical precision and patient safety. This study establishes a benchmark for the robustness of depth estimation and serves as a foundation for developing more resilient surgical support technologies. Code is available at https://github.com/lofrienger/EndoDepthBenchmark.