DPSNet: End-to-end Deep Plane Sweep Stereo
Sunghoon Im, Hae-Gon Jeon, Stephen Lin, In So Kweon

TL;DR
DPSNet is a deep learning model for multiview stereo that integrates traditional geometry-based methods with neural networks, using a differentiable plane sweep approach to produce accurate dense depth maps from multiple images.
Contribution
It introduces a novel end-to-end deep neural network that employs a plane sweep algorithm with differentiable warping for multiview stereo reconstruction.
Findings
Achieves state-of-the-art results on challenging datasets.
Effectively combines traditional geometry with deep learning.
Utilizes a differentiable cost volume for end-to-end training.
Abstract
Multiview stereo aims to reconstruct scene depth from images acquired by a camera under arbitrary motion. Recent methods address this problem through deep learning, which can utilize semantic cues to deal with challenges such as textureless and reflective regions. In this paper, we present a convolutional neural network called DPSNet (Deep Plane Sweep Network) whose design is inspired by best practices of traditional geometry-based approaches for dense depth reconstruction. Rather than directly estimating depth and/or optical flow correspondence from image pairs as done in many previous deep learning methods, DPSNet takes a plane sweep approach that involves building a cost volume from deep features using the plane sweep algorithm, regularizing the cost volume via a context-aware cost aggregation, and regressing the dense depth map from the cost volume. The cost volume is constructed…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Optical measurement and interference techniques · Advanced Image Processing Techniques
