Behavior Imitation for Manipulator Control and Grasping with Deep Reinforcement Learning

TL;DR

This paper introduces a lightweight reinforcement learning approach that uses 3D human pose estimation to enable robotic manipulators to imitate human arm motions from minimal video data, reducing the need for extensive expert datasets.

Contribution

It combines 3D pose estimation with reinforcement learning to simplify motion imitation into a joint angle prediction task, requiring only seconds of video for training.

Findings

Effective imitation of human arm motions from brief videos

Strong generalization to unfamiliar videos

Reduces reliance on costly motion capture data

Abstract

The existing Motion Imitation models typically require expert data obtained through MoCap devices, but the vast amount of training data needed is difficult to acquire, necessitating substantial investments of financial resources, manpower, and time. This project combines 3D human pose estimation with reinforcement learning, proposing a novel model that simplifies Motion Imitation into a prediction problem of joint angle values in reinforcement learning. This significantly reduces the reliance on vast amounts of training data, enabling the agent to learn an imitation policy from just a few seconds of video and exhibit strong generalization capabilities. It can quickly apply the learned policy to imitate human arm motions in unfamiliar videos. The model first extracts skeletal motions of human arms from a given video using 3D human pose estimation. These extracted arm motions are then morphologically retargeted onto a robotic manipulator. Subsequently, the retargeted motions are used to generate reference motions. Finally, these reference motions are used to formulate a reinforcement learning problem, enabling the agent to learn a policy for imitating human arm motions. This project excels at imitation tasks and demonstrates robust transferability, accurately imitating human arm motions from other unfamiliar videos. This project provides a lightweight, convenient, efficient, and accurate Motion Imitation model. While simplifying the complex process of Motion Imitation, it achieves notably outstanding performance.