A Deeper Look into DeepCap
Marc Habermann, Weipeng Xu, Michael Zollhoefer, Gerard Pons-Moll,, Christian Theobalt

TL;DR
This paper introduces a novel deep learning method for monocular human performance capture that does not require 3D ground truth data, outperforming existing methods in quality and robustness.
Contribution
It presents a weakly supervised deep learning approach with a dual-network architecture for dense human performance capture from monocular images.
Findings
Outperforms state-of-the-art in quality and robustness
Operates without 3D ground truth annotations
Effective in dense space-time coherent geometry reconstruction
Abstract
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality. Many previous performance capture approaches either required expensive multi-view setups or did not recover dense space-time coherent geometry with frame-to-frame correspondences. We propose a novel deep learning approach for monocular dense human performance capture. Our method is trained in a weakly supervised manner based on multi-view supervision completely removing the need for training data with 3D ground truth annotations. The network architecture is based on two separate networks that disentangle the task into a pose estimation and a non-rigid surface deformation step. Extensive qualitative and quantitative evaluations show that our approach outperforms the state of the art in terms of quality and robustness. This work is an extended…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Advanced Vision and Imaging · 3D Shape Modeling and Analysis
