DC-GNet: Deep Mesh Relation Capturing Graph Convolution Network for 3D Human Shape Reconstruction

TL;DR

This paper introduces DC-GNet, a graph convolution network that captures deep mesh relations and includes a shape completion task to improve 3D human shape reconstruction from single images, especially under occlusion.

Contribution

The paper presents a novel adaptive matrix for encoding deep relations in mesh vertices and incorporates a shape completion task to handle occlusions, improving robustness.

Findings

Outperforms previous methods on benchmark datasets.

Effectively handles occlusions in outdoor scenes.

Captures subtle and distant relations between mesh nodes.

Abstract

In this paper, we aim to reconstruct a full 3D human shape from a single image. Previous vertex-level and parameter regression approaches reconstruct 3D human shape based on a pre-defined adjacency matrix to encode positive relations between nodes. The deep topological relations for the surface of the 3D human body are not carefully exploited. Moreover, the performance of most existing approaches often suffer from domain gap when handling more occlusion cases in real-world scenes. In this work, we propose a Deep Mesh Relation Capturing Graph Convolution Network, DC-GNet, with a shape completion task for 3D human shape reconstruction. Firstly, we propose to capture deep relations within mesh vertices, where an adaptive matrix encoding both positive and negative relations is introduced. Secondly, we propose a shape completion task to learn prior about various kinds of occlusion cases. Our approach encodes mesh structure from more subtle relations between nodes in a more distant region. Furthermore, our shape completion module alleviates the performance degradation issue in the outdoor scene. Extensive experiments on several benchmarks show that our approach outperforms the previous 3D human pose and shape estimation approaches.