Learning robot motor skills with mixed reality
Eric Rosen, Sreehari Rammohan, Devesh Jha

TL;DR
This paper presents a mixed reality-based framework for teaching robots complex motor skills by integrating demonstrations, constraints, planning, and object info into a unified learning system using Dynamic Movement Primitives.
Contribution
It introduces a novel MR interface for intuitive teaching of diverse motor skills and a sample-efficient learning framework that combines multiple modalities of world knowledge.
Findings
Effective integration of multimodal inputs into DMPs.
Enhanced teaching efficiency with MR interface.
Potential for robots to learn complex tasks more effectively.
Abstract
Mixed Reality (MR) has recently shown great success as an intuitive interface for enabling end-users to teach robots. Related works have used MR interfaces to communicate robot intents and beliefs to a co-located human, as well as developed algorithms for taking multi-modal human input and learning complex motor behaviors. Even with these successes, enabling end-users to teach robots complex motor tasks still poses a challenge because end-user communication is highly task dependent and world knowledge is highly varied. We propose a learning framework where end-users teach robots a) motion demonstrations, b) task constraints, c) planning representations, and d) object information, all of which are integrated into a single motor skill learning framework based on Dynamic Movement Primitives (DMPs). We hypothesize that conveying this world knowledge will be intuitive with an MR interface,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobot Manipulation and Learning · Action Observation and Synchronization · Social Robot Interaction and HRI
