Test3R: Learning to Reconstruct 3D at Test Time

TL;DR

Test3R introduces a simple test-time learning approach that enhances 3D reconstruction accuracy by optimizing for geometric consistency across image pairs, significantly outperforming previous methods with minimal overhead.

Contribution

The paper proposes Test3R, a test-time learning method that improves 3D reconstruction by enforcing cross-pair geometric consistency using self-supervised optimization.

Findings

Outperforms state-of-the-art on 3D reconstruction tasks

Universally applicable with minimal test-time overhead

Significantly improves geometric accuracy and consistency

Abstract

Dense matching methods like DUSt3R regress pairwise pointmaps for 3D reconstruction. However, the reliance on pairwise prediction and the limited generalization capability inherently restrict the global geometric consistency. In this work, we introduce Test3R, a surprisingly simple test-time learning technique that significantly boosts geometric accuracy. Using image triplets ($I_1,I_2,I_3$), Test3R generates reconstructions from pairs ($I_1,I_2$) and ($I_1,I_3$). The core idea is to optimize the network at test time via a self-supervised objective: maximizing the geometric consistency between these two reconstructions relative to the common image $I_1$. This ensures the model produces cross-pair consistent outputs, regardless of the inputs. Extensive experiments demonstrate that our technique significantly outperforms previous state-of-the-art methods on the 3D reconstruction and multi-view depth estimation tasks. Moreover, it is universally applicable and nearly cost-free, making it easily applied to other models and implemented with minimal test-time training overhead and parameter footprint. Code is available at https://github.com/nopQAQ/Test3R.