CeMNet: Self-supervised learning for accurate continuous ego-motion estimation

TL;DR

CeMNet introduces a self-supervised model for continuous ego-motion estimation from video, utilizing a novel motion formulation and segmentation to improve accuracy in dynamic scenes.

Contribution

The paper presents a continuous motion formulation and a segmentation method for robust ego-motion estimation in dynamic environments, advancing self-supervised structure-from-motion techniques.

Findings

State-of-the-art accuracy on ego-motion benchmarks

Superior rotational estimation accuracy

Effective handling of non-rigid scene motions

Abstract

In this paper, we propose a novel self-supervised learning model for estimating continuous ego-motion from video. Our model learns to estimate camera motion by watching RGBD or RGB video streams and determining translational and rotation velocities that correctly predict the appearance of future frames. Our approach differs from other recent work on self-supervised structure-from-motion in its use of a continuous motion formulation and representation of rigid motion fields rather than direct prediction of camera parameters. To make estimation robust in dynamic environments with multiple moving objects, we introduce a simple two-component segmentation process that isolates the rigid background environment from dynamic scene elements. We demonstrate state-of-the-art accuracy of the self-trained model on several benchmark ego-motion datasets and highlight the ability of the model to provide superior rotational accuracy and handling of non-rigid scene motions.