The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTs

TL;DR

This paper introduces a simple ViT-based method for relative pose estimation that incorporates an inductive bias inspired by the Eight-Point Algorithm, achieving competitive results with fewer complexities.

Contribution

It demonstrates how to embed the Eight-Point Algorithm's principles into a ViT, improving pose prediction accuracy especially in limited data scenarios.

Findings

Competitive performance on pose estimation tasks

Significant improvements in limited data regimes

Simplification of existing complex architectures

Abstract

We present a simple baseline for directly estimating the relative pose (rotation and translation, including scale) between two images. Deep methods have recently shown strong progress but often require complex or multi-stage architectures. We show that a handful of modifications can be applied to a Vision Transformer (ViT) to bring its computations close to the Eight-Point Algorithm. This inductive bias enables a simple method to be competitive in multiple settings, often substantially improving over the state of the art with strong performance gains in limited data regimes.