Deep Homography Prediction for Endoscopic Camera Motion Imitation Learning

TL;DR

This paper presents a novel deep learning approach for automating laparoscopic camera motion by learning from retrospective videos, using object motion invariant image registration without geometric assumptions or depth data, enabling direct robotic application.

Contribution

Introduces a new method for laparoscopic camera motion prediction that does not rely on geometric assumptions or scene priors, and demonstrates significant improvements over baselines.

Findings

47% improvement over camera motion continuation baseline

Effective on Cholec80 and HeiChole datasets

Accurately predicts camera motion on AutoLaparo dataset

Abstract

In this work, we investigate laparoscopic camera motion automation through imitation learning from retrospective videos of laparoscopic interventions. A novel method is introduced that learns to augment a surgeon's behavior in image space through object motion invariant image registration via homographies. Contrary to existing approaches, no geometric assumptions are made and no depth information is necessary, enabling immediate translation to a robotic setup. Deviating from the dominant approach in the literature which consist of following a surgical tool, we do not handcraft the objective and no priors are imposed on the surgical scene, allowing the method to discover unbiased policies. In this new research field, significant improvements are demonstrated over two baselines on the Cholec80 and HeiChole datasets, showcasing an improvement of 47% over camera motion continuation. The method is further shown to indeed predict camera motion correctly on the public motion classification labels of the AutoLaparo dataset. All code is made accessible on GitHub.