Compositional 3D Human-Object Neural Animation

TL;DR

This paper introduces a novel neural rendering approach for animating human-object interactions, enabling the creation of new HOI animations with diverse interactions, humans, and objects through compositional neural representations.

Contribution

It proposes a compositional neural radiance field (CC-NeRF) that decomposes human and object interactions for flexible HOI animation, addressing a less explored aspect of HOI rendering.

Findings

The method generalizes well to various novel HOI animation scenarios.

It effectively models and renders dynamic human-object interactions.

The approach enables compositional control over HOI animations.

Abstract

Human-object interactions (HOIs) are crucial for human-centric scene understanding applications such as human-centric visual generation, AR/VR, and robotics. Since existing methods mainly explore capturing HOIs, rendering HOI remains less investigated. In this paper, we address this challenge in HOI animation from a compositional perspective, i.e., animating novel HOIs including novel interaction, novel human and/or novel object driven by a novel pose sequence. Specifically, we adopt neural human-object deformation to model and render HOI dynamics based on implicit neural representations. To enable the interaction pose transferring among different persons and objects, we then devise a new compositional conditional neural radiance field (or CC-NeRF), which decomposes the interdependence between human and object using latent codes to enable compositionally animation control of novel HOIs. Experiments show that the proposed method can generalize well to various novel HOI animation settings. Our project page is https://zhihou7.github.io/CHONA/