Learning Physical Interaction Skills from Human Demonstrations
Tianyu Li, Hengbo Ma, Sehoon Ha, Kwonjoon Lee

TL;DR
This paper presents a novel framework enabling diverse agents to learn physical interaction skills directly from human demonstrations by extracting a transferable interaction representation, facilitating generalization across different morphologies.
Contribution
The introduction of the Embedded Interaction Graph (EIG) as a transferable representation for learning interaction behaviors from human demonstrations across various embodiments.
Findings
Successful learning of interaction behaviors in physics simulations.
Effective generalization across different agent morphologies.
Demonstrated capabilities in multiple interaction scenarios.
Abstract
Learning physical interaction skills, such as dancing, handshaking, or sparring, remains a fundamental challenge for agents operating in human environments, particularly when the agent's morphology differs significantly from that of the demonstrator. Existing approaches often rely on handcrafted objectives or morphological similarity, limiting their capacity for generalization. Here, we introduce a framework that enables agents with diverse embodiments to learn wholebbody interaction behaviors directly from human demonstrations. The framework extracts a compact, transferable representation of interaction dynamics, called the Embedded Interaction Graph (EIG), which captures key spatiotemporal relationships between the interacting agents. This graph is then used as an imitation objective to train control policies in physics-based simulations, allowing the agent to generate motions that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
