SOAR: Self-Occluded Avatar Recovery from a Single Video In the Wild

TL;DR

SOAR introduces a novel method for reconstructing complete human avatars from single videos with self-occlusion, using structural priors and generative diffusion to handle partial observations in unconstrained environments.

Contribution

The paper presents SOAR, a new approach combining structural normal priors and generative diffusion models for full human reconstruction from partial data in the wild.

Findings

Outperforms existing reconstruction methods on various benchmarks.

Achieves comparable results to state-of-the-art approaches.

Effectively reconstructs complete human avatars from partial observations.

Abstract

Self-occlusion is common when capturing people in the wild, where the performer do not follow predefined motion scripts. This challenges existing monocular human reconstruction systems that assume full body visibility. We introduce Self-Occluded Avatar Recovery (SOAR), a method for complete human reconstruction from partial observations where parts of the body are entirely unobserved. SOAR leverages structural normal prior and generative diffusion prior to address such an ill-posed reconstruction problem. For structural normal prior, we model human with an reposable surfel model with well-defined and easily readable shapes. For generative diffusion prior, we perform an initial reconstruction and refine it using score distillation. On various benchmarks, we show that SOAR performs favorably than state-of-the-art reconstruction and generation methods, and on-par comparing to concurrent works. Additional video results and code are available at https://soar-avatar.github.io/.