MoRe-ERL: Learning Motion Residuals using Episodic Reinforcement Learning

TL;DR

MoRe-ERL introduces a novel framework combining episodic reinforcement learning and residual learning to improve trajectory planning, achieving better efficiency, safety, and real-world applicability in robotic tasks.

Contribution

The paper presents a general residual learning framework integrated with ERL that refines preplanned trajectories for enhanced robot motion planning.

Findings

Residual learning outperforms training from scratch in sample efficiency.

Policies trained in simulation transfer effectively to real-world systems.

Framework is adaptable to various ERL methods and motion generators.

Abstract

We propose MoRe-ERL, a framework that combines Episodic Reinforcement Learning (ERL) and residual learning, which refines preplanned reference trajectories into safe, feasible, and efficient task-specific trajectories. This framework is general enough to incorporate into arbitrary ERL methods and motion generators seamlessly. MoRe-ERL identifies trajectory segments requiring modification while preserving critical task-related maneuvers. Then it generates smooth residual adjustments using B-Spline-based movement primitives to ensure adaptability to dynamic task contexts and smoothness in trajectory refinement. Experimental results demonstrate that residual learning significantly outperforms training from scratch using ERL methods, achieving superior sample efficiency and task performance. Hardware evaluations further validate the framework, showing that policies trained in simulation can be directly deployed in real-world systems, exhibiting a minimal sim-to-real gap.