Shape Animation with Combined Captured and Simulated Dynamics

TL;DR

This paper introduces a volumetric animation framework that combines real captured performances with virtual physics to enable novel effects like distortion, erosion, and collision-based reanimation, using a unified Voronoi tessellation representation.

Contribution

The authors develop a new volumetric animation pipeline using Centroidal Voronoi tessellations that seamlessly integrates real capture data with virtual physics for advanced effects.

Findings

Effective volumetric reanimation of real captured performances.

Ability to apply complex virtual effects like erosion and morphing.

Flexible representation that supports props and scenery integration.

Abstract

We present a novel volumetric animation generation framework to create new types of animations from raw 3D surface or point cloud sequence of captured real performances. The framework considers as input time incoherent 3D observations of a moving shape, and is thus particularly suitable for the output of performance capture platforms. In our system, a suitable virtual representation of the actor is built from real captures that allows seamless combination and simulation with virtual external forces and objects, in which the original captured actor can be reshaped, disassembled or reassembled from user-specified virtual physics. Instead of using the dominant surface-based geometric representation of the capture, which is less suitable for volumetric effects, our pipeline exploits Centroidal Voronoi tessellation decompositions as unified volumetric representation of the real captured actor, which we show can be used seamlessly as a building block for all processing stages, from capture and tracking to virtual physic simulation. The representation makes no human specific assumption and can be used to capture and re-simulate the actor with props or other moving scenery elements. We demonstrate the potential of this pipeline for virtual reanimation of a real captured event with various unprecedented volumetric visual effects, such as volumetric distortion, erosion, morphing, gravity pull, or collisions.