Learning Dense Wide Baseline Stereo Matching for People

TL;DR

This paper introduces a deep learning framework for dense stereo matching of people from wide baseline image pairs, utilizing synthetic data and stereo constraints to improve reconstruction accuracy.

Contribution

It presents a novel synthetic dataset (S2P2) and a learning framework that adapts human-specific features for wide baseline stereo matching, outperforming existing methods.

Findings

Improved stereo reconstruction accuracy on challenging datasets.

Effective use of synthetic data for real-world stereo matching.

Enhanced performance in wide baseline human stereo reconstruction.

Abstract

Existing methods for stereo work on narrow baseline image pairs giving limited performance between wide baseline views. This paper proposes a framework to learn and estimate dense stereo for people from wide baseline image pairs. A synthetic people stereo patch dataset (S2P2) is introduced to learn wide baseline dense stereo matching for people. The proposed framework not only learns human specific features from synthetic data but also exploits pooling layer and data augmentation to adapt to real data. The network learns from the human specific stereo patches from the proposed dataset for wide-baseline stereo estimation. In addition to patch match learning, a stereo constraint is introduced in the framework to solve wide baseline stereo reconstruction of humans. Quantitative and qualitative performance evaluation against state-of-the-art methods of proposed method demonstrates improved wide baseline stereo reconstruction on challenging datasets. We show that it is possible to learn stereo matching from synthetic people dataset and improve performance on real datasets for stereo reconstruction of people from narrow and wide baseline stereo data.