Object Properties Inferring from and Transfer for Human Interaction Motions

TL;DR

This paper introduces a method to infer latent object properties such as weight and fragility from human skeletal motion during interactions, enabling better understanding and synthesis of human-object interactions.

Contribution

It presents a fine-grained action recognition approach that infers object properties solely from human motion data, without visual object information.

Findings

Object properties can be inferred from skeletal motion alone.

Interaction motions are highly correlated with object properties.

The method enables new motion synthesis possibilities.

Abstract

Humans regularly interact with their surrounding objects. Such interactions often result in strongly correlated motion between humans and the interacting objects. We thus ask: "Is it possible to infer object properties from skeletal motion alone, even without seeing the interacting object itself?" In this paper, we present a fine-grained action recognition method that learns to infer such latent object properties from human interaction motion alone. This inference allows us to disentangle the motion from the object property and transfer object properties to a given motion. We collected a large number of videos and 3D skeletal motions of the performing actors using an inertial motion capture device. We analyze similar actions and learn subtle differences among them to reveal latent properties of the interacting objects. In particular, we learn to identify the interacting object, by estimating its weight, or its fragility or delicacy. Our results clearly demonstrate that the interaction motions and interacting objects are highly correlated and indeed relative object latent properties can be inferred from the 3D skeleton sequences alone, leading to new synthesis possibilities for human interaction motions. Dataset will be available at http://vcc.szu.edu.cn/research/2020/IT.