InterAct: Capture and Modelling of Realistic, Expressive and Interactive Activities between Two Persons in Daily Scenarios
Yinghao Huang, Leo Ho, Dafei Qin, Mingyi Shi, and Taku Komura

TL;DR
This paper introduces InterAct, a new dataset and a diffusion model approach for capturing and modeling realistic, expressive, and interactive behaviors between two persons in daily scenarios, focusing on long-duration, objective-driven interactions.
Contribution
It presents a novel dataset capturing multi-modal data of two-person interactions and a diffusion model for directly estimating interactive motions from audio.
Findings
The dataset includes 241 multi-modal motion sequences of two persons.
The diffusion model effectively estimates interactive motions from audio alone.
The approach advances modeling of long-duration, expressive two-person interactions.
Abstract
We address the problem of accurate capture and expressive modelling of interactive behaviors happening between two persons in daily scenarios. Different from previous works which either only consider one person or focus on conversational gestures, we propose to simultaneously model the activities of two persons, and target objective-driven, dynamic, and coherent interactions which often span long duration. To this end, we capture a new dataset dubbed InterAct, which is composed of 241 motion sequences where two persons perform a realistic scenario over the whole sequence. The audios, body motions, and facial expressions of both persons are all captured in our dataset. We also demonstrate the first diffusion model based approach that directly estimates the interactive motions between two persons from their audios alone. All the data and code will be available at:…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimedia Communication and Technology
MethodsFocus · Diffusion
