Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space

TL;DR

This paper introduces AFORCE, a hierarchical action space combining reinforcement learning and adaptive control, enabling more efficient, safe, and energy-conscious robotic manipulation in real-world tasks.

Contribution

The paper proposes a novel bio-inspired hierarchical action space called AFORCE that integrates RL and adaptive control for improved robotic manipulation.

Findings

AFORCE significantly improves sample efficiency in manipulation tasks.

It reduces energy consumption during robotic operations.

The approach enhances safety in contact-rich interactions.

Abstract

Intelligent agents must be able to think fast and slow to perform elaborate manipulation tasks. Reinforcement Learning (RL) has led to many promising results on a range of challenging decision-making tasks. However, in real-world robotics, these methods still struggle, as they require large amounts of expensive interactions and have slow feedback loops. On the other hand, fast human-like adaptive control methods can optimize complex robotic interactions, yet fail to integrate multimodal feedback needed for unstructured tasks. In this work, we propose to factor the learning problem in a hierarchical learning and adaption architecture to get the best of both worlds. The framework consists of two components, a slow reinforcement learning policy optimizing the task strategy given multimodal observations, and a fast, real-time adaptive control policy continuously optimizing the motion, stability, and effort of the manipulator. We combine these components through a bio-inspired action space that we call AFORCE. We demonstrate the new action space on a contact-rich manipulation task on real hardware and evaluate its performance on three simulated manipulation tasks. Our experiments show that AFORCE drastically improves sample efficiency while reducing energy consumption and improving safety.