Self-Supervised Correspondence Estimation via Multiview Registration

TL;DR

This paper introduces a self-supervised method for correspondence estimation in RGB-D videos that leverages multiview registration to capture long-range consistency, improving over prior methods limited to close-by frames.

Contribution

It proposes a novel self-supervised multiview registration approach that enhances correspondence estimation by utilizing long-range frame relationships through SE(3) synchronization.

Findings

Performs on par with supervised methods in indoor scene experiments.

Effectively captures long-range correspondences using multiview registration.

Improves diversity and difficulty of sampled pairs for learning.

Abstract

Video provides us with the spatio-temporal consistency needed for visual learning. Recent approaches have utilized this signal to learn correspondence estimation from close-by frame pairs. However, by only relying on close-by frame pairs, those approaches miss out on the richer long-range consistency between distant overlapping frames. To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences. Our approach combines pairwise correspondence estimation and registration with a novel SE(3) transformation synchronization algorithm. Our key insight is that self-supervised multiview registration allows us to obtain correspondences over longer time frames; increasing both the diversity and difficulty of sampled pairs. We evaluate our approach on indoor scenes for correspondence estimation and RGB-D pointcloud registration and find that we perform on-par with supervised approaches.