KD-MVS: Knowledge Distillation Based Self-supervised Learning for Multi-view Stereo

TL;DR

KD-MVS introduces a self-supervised multi-view stereo training pipeline using knowledge distillation, enabling the student model to outperform supervised methods without requiring large-scale ground-truth depth data.

Contribution

This paper presents a novel self-supervised training pipeline for MVS based on knowledge distillation, which improves reconstruction quality without large-scale ground-truth data.

Findings

Student model outperforms its teacher significantly.

Method can surpass supervised MVS methods.

Effective knowledge transfer via probabilistic distillation.

Abstract

Supervised multi-view stereo (MVS) methods have achieved remarkable progress in terms of reconstruction quality, but suffer from the challenge of collecting large-scale ground-truth depth. In this paper, we propose a novel self-supervised training pipeline for MVS based on knowledge distillation, termed KD-MVS, which mainly consists of self-supervised teacher training and distillation-based student training. Specifically, the teacher model is trained in a self-supervised fashion using both photometric and featuremetric consistency. Then we distill the knowledge of the teacher model to the student model through probabilistic knowledge transferring. With the supervision of validated knowledge, the student model is able to outperform its teacher by a large margin. Extensive experiments performed on multiple datasets show our method can even outperform supervised methods.