MVSMamba: Multi-View Stereo with State Space Model

TL;DR

MVSMamba introduces a novel multi-view stereo network leveraging the Mamba architecture for efficient global feature aggregation, outperforming existing methods in accuracy and computational efficiency on standard benchmarks.

Contribution

This paper presents the first Mamba-based MVS network with a dynamic module for efficient multi-view feature interaction and global aggregation, improving performance and efficiency.

Findings

Outperforms state-of-the-art methods on DTU and Tanks-and-Temples datasets.

Achieves better accuracy with lower computational overhead.

Demonstrates the effectiveness of Mamba architecture in MVS tasks.

Abstract

Robust feature representations are essential for learning-based Multi-View Stereo (MVS), which relies on accurate feature matching. Recent MVS methods leverage Transformers to capture long-range dependencies based on local features extracted by conventional feature pyramid networks. However, the quadratic complexity of Transformer-based MVS methods poses challenges to balance performance and efficiency. Motivated by the global modeling capability and linear complexity of the Mamba architecture, we propose MVSMamba, the first Mamba-based MVS network. MVSMamba enables efficient global feature aggregation with minimal computational overhead. To fully exploit Mamba's potential in MVS, we propose a Dynamic Mamba module (DM-module) based on a novel reference-centered dynamic scanning strategy, which enables: (1) Efficient intra- and inter-view feature interaction from the reference to source views, (2) Omnidirectional multi-view feature representations, and (3) Multi-scale global feature aggregation. Extensive experimental results demonstrate MVSMamba outperforms state-of-the-art MVS methods on the DTU dataset and the Tanks-and-Temples benchmark with both superior performance and efficiency. The source code is available at https://github.com/JianfeiJ/MVSMamba.