MVS-TTA: Test-Time Adaptation for Multi-View Stereo via Meta-Auxiliary Learning

TL;DR

MVS-TTA introduces a novel test-time adaptation framework for multi-view stereo that combines meta-learning with self-supervised auxiliary tasks to improve model generalization across diverse datasets.

Contribution

It is the first to integrate optimization-based test-time adaptation into learning-based MVS using meta-learning, enhancing adaptability and performance.

Findings

Consistently improves MVS performance on standard datasets.

Effective in cross-dataset generalization scenarios.

Model-agnostic and easy to integrate with existing MVS methods.

Abstract

Recent learning-based multi-view stereo (MVS) methods are data-driven and have achieved remarkable progress due to large-scale training data and advanced architectures. However, their generalization remains sub-optimal due to fixed model parameters trained on limited training data distributions. In contrast, optimization-based methods enable scene-specific adaptation but lack scalability and require costly per-scene optimization. In this paper, we propose MVS-TTA, an efficient test-time adaptation (TTA) framework that enhances the adaptability of learning-based MVS methods by bridging these two paradigms. Specifically, MVS-TTA employs a self-supervised, cross-view consistency loss as an auxiliary task to guide inference-time adaptation. We introduce a meta-auxiliary learning strategy to train the model to benefit from auxiliary-task-based updates explicitly. Our framework is model-agnostic and can be applied to a wide range of MVS methods with minimal architectural changes. Extensive experiments on standard datasets (DTU, BlendedMVS) and a challenging cross-dataset generalization setting demonstrate that MVS-TTA consistently improves performance, even when applied to state-of-the-art MVS models. To our knowledge, this is the first attempt to integrate optimization-based test-time adaptation into learning-based MVS using meta-learning. The code will be available at https://github.com/mart87987-svg/MVS-TTA.