DepthPilot: From Controllability to Interpretability in Colonoscopy Video Generation

TL;DR

DepthPilot introduces an interpretable colonoscopy video generation framework that aligns with physical priors and clinical features, enhancing trustworthiness and clinical utility.

Contribution

It is the first framework to incorporate explicit geometric grounding and adaptive nonlinear modeling for realistic, interpretable medical video synthesis.

Findings

Achieves FID scores below 15 on all benchmarks.

Ranks first in clinician assessments for interpretability.

Produces physically consistent and clinically relevant videos.

Abstract

Controllable medical video generation has achieved remarkable progress, but it still lacks interpretability, which requires the alignment of generated contents with physical priors and faithful clinical manifestations. To push the boundaries from mere controllability to interpretability, we propose DepthPilot, the first interpretable framework for colonoscopy video generation. This work takes a step toward trustworthy generation through two synergistic paradigms. To achieve explicit geometric grounding, DepthPilot devises a prior distribution alignment strategy, injecting depth constraints into the diffusion backbone via parameter-efficient fine-tuning to ensure anatomical fidelity. To enhance intrinsic nonlinear modeling under these geometric constraints, DepthPilot employs an adaptive spline denoising module, replacing fixed linear weights with learnable spline functions to capture complex spatio-temporal dynamics. Extensive evaluations across three public datasets and in-house clinical data confirm DepthPilot's robust ability to produce physically consistent videos. It achieves FID scores below 15 across all benchmarks and ranks first in clinician assessments, bridging the gap between "visually realistic" and "clinically interpretable". Moreover, DepthPilot-generated videos are expected to enable reliable 3D reconstruction, facilitating surgical navigation and blind region identification, and serve as a foundation toward the colorectal world model.