Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motion
Vladimir Guzov, Julian Chibane, Riccardo Marin, Yannan He, Yunus, Saracoglu, Torsten Sattler, Gerard Pons-Moll

TL;DR
This paper introduces iReplica, a novel method for modeling human-object interactions and scene changes using simple ego-centric sensors, enabling realistic prediction of interactions without external tracking systems.
Contribution
We develop a new approach combining visual localization and contact-based reasoning from IMU data for egocentric interaction modeling, filling a gap in current research.
Findings
Human-scene contacts can be predicted from pose sequences.
The method enables realistic interaction modeling without external cameras.
iReplica advances egocentric capture for AR/VR and robotics applications.
Abstract
Our world is not static and humans naturally cause changes in their environments through interactions, e.g., opening doors or moving furniture. Modeling changes caused by humans is essential for building digital twins, e.g., in the context of shared physical-virtual spaces (metaverses) and robotics. In order for widespread adoption of such emerging applications, the sensor setup used to capture the interactions needs to be inexpensive and easy-to-use for non-expert users. I.e., interactions should be captured and modeled by simple ego-centric sensors such as a combination of cameras and IMU sensors, not relying on any external cameras or object trackers. Yet, to the best of our knowledge, no work tackling the challenging problem of modeling human-scene interactions via such an ego-centric sensor setup exists. This paper closes this gap in the literature by developing a novel approach…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Video Surveillance and Tracking Methods · Advanced Vision and Imaging
