QuadSync: Quadrifocal Tensor Synchronization via Tucker Decomposition

TL;DR

This paper introduces a novel Tucker decomposition-based framework for recovering multiple cameras from quadrifocal tensors, enabling effective synchronization and demonstrating the value of higher-order information.

Contribution

It presents the first synchronization algorithm for quadrifocal tensors using Tucker decomposition and relates it to trifocal and bifocal tensors, advancing structure from motion techniques.

Findings

The Tucker decomposition approach successfully recovers camera configurations.

The joint synchronization of quadrifocal, trifocal, and bifocal tensors improves accuracy.

Numerical experiments confirm the effectiveness of the proposed methods.

Abstract

In structure from motion, quadrifocal tensors capture more information than their pairwise counterparts (essential matrices), yet they have often been thought of as impractical and only of theoretical interest. In this work, we challenge such beliefs by providing a new framework to recover $n$ cameras from the corresponding collection of quadrifocal tensors. We form the block quadrifocal tensor and show that it admits a Tucker decomposition whose factor matrices are the stacked camera matrices, and which thus has a multilinear rank of (4,~4,~4,~4) independent of $n$. We develop the first synchronization algorithm for quadrifocal tensors, using Tucker decomposition, alternating direction method of multipliers, and iteratively reweighted least squares. We further establish relationships between the block quadrifocal, trifocal, and bifocal tensors, and introduce an algorithm that jointly synchronizes these three entities. Numerical experiments demonstrate the effectiveness of our methods on modern datasets, indicating the potential and importance of using higher-order information in synchronization.