Learning to Adapt Multi-View Stereo by Self-Supervision

TL;DR

This paper introduces a self-supervised, adaptive multi-view stereo learning method using meta-learning to improve domain generalization in 3D scene reconstruction.

Contribution

It proposes a novel meta-learning based approach that enables multi-view stereo models to adapt to new domains without ground truth data.

Findings

Effective domain adaptation demonstrated in experiments

Improved reconstruction accuracy in new environments

Self-supervised learning reduces reliance on labeled data

Abstract

3D scene reconstruction from multiple views is an important classical problem in computer vision. Deep learning based approaches have recently demonstrated impressive reconstruction results. When training such models, self-supervised methods are favourable since they do not rely on ground truth data which would be needed for supervised training and is often difficult to obtain. Moreover, learned multi-view stereo reconstruction is prone to environment changes and should robustly generalise to different domains. We propose an adaptive learning approach for multi-view stereo which trains a deep neural network for improved adaptability to new target domains. We use model-agnostic meta-learning (MAML) to train base parameters which, in turn, are adapted for multi-view stereo on new domains through self-supervised training. Our evaluations demonstrate that the proposed adaptation method is effective in learning self-supervised multi-view stereo reconstruction in new domains.