Self-supervised Pretraining and Finetuning for Monocular Depth and Visual Odometry

TL;DR

This paper introduces a self-supervised transformer-based approach for monocular depth and visual odometry estimation, achieving state-of-the-art results across diverse datasets without relying on annotated data.

Contribution

It proposes a two-step self-supervised training framework using cross-view completion pretraining and finetuning, leveraging transformers for improved depth and odometry estimation.

Findings

Outperforms existing methods on six benchmark datasets.

Achieves superior depth prediction accuracy.

Effective across static, dynamic, indoor, outdoor, synthetic, and real images.

Abstract

For the task of simultaneous monocular depth and visual odometry estimation, we propose learning self-supervised transformer-based models in two steps. Our first step consists in a generic pretraining to learn 3D geometry, using cross-view completion objective (CroCo), followed by self-supervised finetuning on non-annotated videos. We show that our self-supervised models can reach state-of-the-art performance 'without bells and whistles' using standard components such as visual transformers, dense prediction transformers and adapters. We demonstrate the effectiveness of our proposed method by running evaluations on six benchmark datasets, both static and dynamic, indoor and outdoor, with synthetic and real images. For all datasets, our method outperforms state-of-the-art methods, in particular for depth prediction task.