The Drunkard's Odometry: Estimating Camera Motion in Deforming Scenes

TL;DR

This paper introduces a new synthetic dataset and a deformable odometry method for estimating camera motion in scenes with non-rigid deformations, addressing challenges in visual navigation and reconstruction.

Contribution

It presents the first large-scale dataset with ground truth for deformable scenes and a novel odometry algorithm that separates rigid motion from non-rigid deformations.

Findings

The dataset enables robust evaluation of deformable odometry methods.

The proposed method effectively decomposes optical flow into rigid and non-rigid components.

A new tracking error metric improves evaluation without ground truth.

Abstract

Estimating camera motion in deformable scenes poses a complex and open research challenge. Most existing non-rigid structure from motion techniques assume to observe also static scene parts besides deforming scene parts in order to establish an anchoring reference. However, this assumption does not hold true in certain relevant application cases such as endoscopies. Deformable odometry and SLAM pipelines, which tackle the most challenging scenario of exploratory trajectories, suffer from a lack of robustness and proper quantitative evaluation methodologies. To tackle this issue with a common benchmark, we introduce the Drunkard's Dataset, a challenging collection of synthetic data targeting visual navigation and reconstruction in deformable environments. This dataset is the first large set of exploratory camera trajectories with ground truth inside 3D scenes where every surface exhibits non-rigid deformations over time. Simulations in realistic 3D buildings lets us obtain a vast amount of data and ground truth labels, including camera poses, RGB images and depth, optical flow and normal maps at high resolution and quality. We further present a novel deformable odometry method, dubbed the Drunkard's Odometry, which decomposes optical flow estimates into rigid-body camera motion and non-rigid scene deformations. In order to validate our data, our work contains an evaluation of several baselines as well as a novel tracking error metric which does not require ground truth data. Dataset and code: https://davidrecasens.github.io/TheDrunkard'sOdometry/