Reinforcement Learning for Follow-the-Leader Robotic Endoscopic Navigation via Synthetic Data

TL;DR

This paper introduces a reinforcement learning-based autonomous endoscopic navigation system that uses synthetic data and monocular depth estimation to improve lumen tracking and reduce wall contact, enhancing safety and comfort.

Contribution

It presents a novel deep reinforcement learning framework guided by synthetic data and a geometry-aware reward for follow-the-leader endoscopic navigation, with improved depth accuracy and robustness.

Findings

Depth accuracy improved by 39.2% over original model

Navigation J-index reduced by 0.67, indicating better performance

Synthetic data and simulation environment effectively train autonomous navigation

Abstract

Autonomous navigation is crucial for both medical and industrial endoscopic robots, enabling safe and efficient exploration of narrow tubular environments without continuous human intervention, where avoiding contact with the inner walls has been a longstanding challenge for prior approaches. We present a follow-the-leader endoscopic robot based on a flexible continuum structure designed to minimize contact between the endoscope body and intestinal walls, thereby reducing patient discomfort. To achieve this objective, we propose a vision-based deep reinforcement learning framework guided by monocular depth estimation. A realistic intestinal simulation environment was constructed in \textit{NVIDIA Omniverse} to train and evaluate autonomous navigation strategies. Furthermore, thousands of synthetic intraluminal images were generated using NVIDIA Replicator to fine-tune the Depth Anything model, enabling dense three-dimensional perception of the intestinal environment with a single monocular camera. Subsequently, we introduce a geometry-aware reward and penalty mechanism to enable accurate lumen tracking. Compared with the original Depth Anything model, our method improves $\delta_{1}$ depth accuracy by 39.2% and reduces the navigation J-index by 0.67 relative to the second-best method, demonstrating the robustness and effectiveness of the proposed approach.