Impedance Primitive-augmented Hierarchical Reinforcement Learning for Sequential Tasks

TL;DR

This paper introduces an impedance primitive-augmented hierarchical reinforcement learning framework that enhances robotic manipulation in contact tasks by integrating variable stiffness control, adaptive stiffness adjustment, and affordance coupling, leading to improved learning efficiency and success rates.

Contribution

The paper proposes a novel hierarchical RL framework with impedance primitives, enabling adaptive stiffness control and efficient exploration for contact-rich manipulation tasks.

Findings

Improved learning efficiency over state-of-the-art methods.

Enhanced success rates in various manipulation tasks.

Effective sim2real transfer demonstrated in real-world tests.

Abstract

This paper presents an Impedance Primitive-augmented hierarchical reinforcement learning framework for efficient robotic manipulation in sequential contact tasks. We leverage this hierarchical structure to sequentially execute behavior primitives with variable stiffness control capabilities for contact tasks. Our proposed approach relies on three key components: an action space enabling variable stiffness control, an adaptive stiffness controller for dynamic stiffness adjustments during primitive execution, and affordance coupling for efficient exploration while encouraging compliance. Through comprehensive training and evaluation, our framework learns efficient stiffness control capabilities and demonstrates improvements in learning efficiency, compositionality in primitive selection, and success rates compared to the state-of-the-art. The training environments include block lifting, door opening, object pushing, and surface cleaning. Real world evaluations further confirm the framework's sim2real capability. This work lays the foundation for more adaptive and versatile robotic manipulation systems, with potential applications in more complex contact-based tasks.