Point‐Guided Latent Diffusion Model for Novel View Synthesis in Laparoscopic Liver Surgery

TL;DR

This paper introduces a new method for generating detailed surgical views from limited laparoscopic footage, improving accuracy and situational awareness during liver surgery.

Contribution

A point-guided latent diffusion model is proposed for novel view synthesis in laparoscopic liver surgery, using geometric cues and adaptive camera planning.

Findings

The method outperforms existing approaches in terms of PNSR, SSIM, and LPIPS metrics on the P2ILF dataset.

It effectively handles occlusions and shape deformation through adaptive camera trajectory planning and a spatial-transformer enhanced decoder.

The approach generates anatomically consistent views, enhancing surgical scene reconstruction and training.

Abstract

Despite recent progress in diffusion‐based video synthesis, synthesizing accurate novel views from sparse input frames in laparoscopic liver surgery remains challenging due to occlusions, complex shape of anatomical structures and limited field of views. We propose point‐guided latent diffusion model, specifically designed for generating high‐quality intermediate frames in laparoscopic liver surgery from only the first and last video frames. Our method leverages the powerful generative capability of latent diffusion models combined with geometric cues from 3D point clouds reconstructed via dense stereo matching. To robustly handle occlusions and shape deformation, we use an adaptive camera trajectory planning strategy based on next‐best‐view algorithms. Furthermore, we introduce a spatial‐transformer enhanced decoder to effectively preserve detailed anatomical features from reference frames and minimize visual artefacts in generated views. Extensive experiments on the clinically relevant P2ILF challenge dataset validate our method's effectiveness and superior performance in producing visually coherent and structurally accurate novel views, highlighting its ability for enhancing the quality of surgical scene reconstruction.

This approach enables novel view synthesis from sparse laparoscopic inputs, reconstructing anatomically consistent views of regions not directly captured. It improves intra‐operative situational awareness, supports surgical training by generating multiple perspectives from limited recordings. The evaluation results demonstrate that the proposed approach leads to the performance of novel synthesis compared with SVD, CF‐3DGS and ViewCrafter in terms of PNSR, SSIM and LPIPS metrics.

Figures