Learning Dynamic Manipulation Skills from Haptic-Play
Taeyoon Lee, Donghyun Sung, Kyoungyeon Choi, Choongin Lee, Changwoo, Park, Keunjun Choi

TL;DR
This paper introduces a data-driven approach for learning dynamic manipulation skills from offline teleoperated play data, enabling robots to perform complex tasks through learned skill representations and model-based planning.
Contribution
It presents a novel two-stage generative modeling framework for learning goal-conditioned policies and skill dynamics from teleoperation data, facilitating robust online and offline planning.
Findings
Successful real-world dual-arm manipulation of boxes to random targets
Effective composition of force-controlled dynamic skills in real-time
Demonstrated robustness in both simulated and real environments
Abstract
In this paper, we propose a data-driven skill learning approach to solve highly dynamic manipulation tasks entirely from offline teleoperated play data. We use a bilateral teleoperation system to continuously collect a large set of dexterous and agile manipulation behaviors, which is enabled by providing direct force feedback to the operator. We jointly learn the state conditional latent skill distribution and skill decoder network in the form of goal-conditioned policy and skill conditional state transition dynamics using a two-stage generative modeling framework. This allows one to perform robust model-based planning, both online and offline planning methods, in the learned skill-space to accomplish any given downstream tasks at test time. We provide both simulated and real-world dual-arm box manipulation experiments showing that a sequence of force-controlled dynamic manipulation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobot Manipulation and Learning · Teleoperation and Haptic Systems · Human Pose and Action Recognition
