TL;DR
This paper introduces a novel 3D reasoning framework for synthesizing human actions from new viewpoints, combining mesh estimation, texture transfer, and a context-based generator to improve visual consistency.
Contribution
It proposes a new method that estimates 3D meshes, propagates textures, and uses a context-based generator for improved novel-view human action synthesis.
Findings
Effective in synthesizing realistic novel-view human actions
Outperforms existing methods on NTU RGB+D dataset
Code and resources publicly available
Abstract
Novel-View Human Action Synthesis aims to synthesize the movement of a body from a virtual viewpoint, given a video from a real viewpoint. We present a novel 3D reasoning to synthesize the target viewpoint. We first estimate the 3D mesh of the target body and transfer the rough textures from the 2D images to the mesh. As this transfer may generate sparse textures on the mesh due to frame resolution or occlusions. We produce a semi-dense textured mesh by propagating the transferred textures both locally, within local geodesic neighborhoods, and globally, across symmetric semantic parts. Next, we introduce a context-based generator to learn how to correct and complete the residual appearance information. This allows the network to independently focus on learning the foreground and background synthesis tasks. We validate the proposed solution on the public NTU RGB+D dataset. The code and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
