A Graph Neural Network Approach for Temporal Mesh Blending and Correspondence

TL;DR

This paper introduces a self-supervised graph neural network framework for temporal mesh blending and correspondence, capable of handling unaligned meshes and producing realistic human body deformations.

Contribution

It presents Red-Blue MPNN, a novel GNN architecture with a conditional refinement scheme for mesh correspondence and blending in a self-supervised setting.

Findings

Successfully estimates mesh correspondence without supervision.

Generates realistic human body deformations from complex inputs.

Outperforms existing methods on synthetic motion capture datasets.

Abstract

We have proposed a self-supervised deep learning framework for solving the mesh blending problem in scenarios where the meshes are not in correspondence. To solve this problem, we have developed Red-Blue MPNN, a novel graph neural network that processes an augmented graph to estimate the correspondence. We have designed a novel conditional refinement scheme to find the exact correspondence when certain conditions are satisfied. We further develop a graph neural network that takes the aligned meshes and the time value as input and fuses this information to process further and generate the desired result. Using motion capture datasets and human mesh designing software, we create a large-scale synthetic dataset consisting of temporal sequences of human meshes in motion. Our results demonstrate that our approach generates realistic deformation of body parts given complex inputs.