Learning Dense UV Completion for Human Mesh Recovery

TL;DR

This paper introduces DIMR, a two-stage dense inpainting approach for human mesh recovery from single images, effectively handling occlusions by completing features on a structured UV map.

Contribution

The paper presents a novel dense inpainting framework with an attention-based feature completion module and a training procedure guiding feature learning from unoccluded data.

Findings

Outperforms prior SOTA methods on heavily occluded images

Achieves comparable results on standard benchmarks like 3DPW

Demonstrates superior performance in occlusion scenarios

Abstract

Human mesh reconstruction from a single image is challenging in the presence of occlusion, which can be caused by self, objects, or other humans. Existing methods either fail to separate human features accurately or lack proper supervision for feature completion. In this paper, we propose Dense Inpainting Human Mesh Recovery (DIMR), a two-stage method that leverages dense correspondence maps to handle occlusion. Our method utilizes a dense correspondence map to separate visible human features and completes human features on a structured UV map dense human with an attention-based feature completion module. We also design a feature inpainting training procedure that guides the network to learn from unoccluded features. We evaluate our method on several datasets and demonstrate its superior performance under heavily occluded scenarios compared to other methods. Extensive experiments show that our method obviously outperforms prior SOTA methods on heavily occluded images and achieves comparable results on the standard benchmarks (3DPW).