InterAct: Capture and Modelling of Realistic, Expressive and Interactive Activities between Two Persons in Daily Scenarios

TL;DR

This paper introduces InterAct, a new dataset and a diffusion model approach for capturing and modeling realistic, expressive, and interactive behaviors between two persons in daily scenarios, focusing on long-duration, objective-driven interactions.

Contribution

It presents a novel dataset capturing multi-modal data of two-person interactions and a diffusion model for directly estimating interactive motions from audio.

Findings

The dataset includes 241 multi-modal motion sequences of two persons.

The diffusion model effectively estimates interactive motions from audio alone.

The approach advances modeling of long-duration, expressive two-person interactions.

Abstract

We address the problem of accurate capture and expressive modelling of interactive behaviors happening between two persons in daily scenarios. Different from previous works which either only consider one person or focus on conversational gestures, we propose to simultaneously model the activities of two persons, and target objective-driven, dynamic, and coherent interactions which often span long duration. To this end, we capture a new dataset dubbed InterAct, which is composed of 241 motion sequences where two persons perform a realistic scenario over the whole sequence. The audios, body motions, and facial expressions of both persons are all captured in our dataset. We also demonstrate the first diffusion model based approach that directly estimates the interactive motions between two persons from their audios alone. All the data and code will be available at: https://hku-cg.github.io/interact.