WS-SfMLearner: Self-supervised Monocular Depth and Ego-motion Estimation on Surgical Videos with Unknown Camera Parameters

TL;DR

This paper introduces WS-SfMLearner, a self-supervised system for estimating depth, ego-motion, and camera intrinsic parameters in surgical videos without requiring known camera parameters or ground truth data.

Contribution

It presents a novel cost-volume-based supervision approach enabling accurate prediction of depth, ego-motion, and camera intrinsics in challenging surgical video environments.

Findings

Improved accuracy of camera parameter estimation

Enhanced depth and ego-motion prediction performance

Effective self-supervised learning without known camera parameters

Abstract

Depth estimation in surgical video plays a crucial role in many image-guided surgery procedures. However, it is difficult and time consuming to create depth map ground truth datasets in surgical videos due in part to inconsistent brightness and noise in the surgical scene. Therefore, building an accurate and robust self-supervised depth and camera ego-motion estimation system is gaining more attention from the computer vision community. Although several self-supervision methods alleviate the need for ground truth depth maps and poses, they still need known camera intrinsic parameters, which are often missing or not recorded. Moreover, the camera intrinsic prediction methods in existing works depend heavily on the quality of datasets. In this work, we aimed to build a self-supervised depth and ego-motion estimation system which can predict not only accurate depth maps and camera pose, but also camera intrinsic parameters. We proposed a cost-volume-based supervision manner to give the system auxiliary supervision for camera parameters prediction. The experimental results showed that the proposed method improved the accuracy of estimated camera parameters, ego-motion, and depth estimation.