Cascaded Compositional Residual Learning for Complex Interactive Behaviors

TL;DR

The paper introduces CCRL, a framework that learns complex interactive behaviors by composing and refining existing control policies, enabling robots to perform diverse tasks with style consistency and transferability.

Contribution

It presents a novel recursive policy composition method that leverages pre-trained policies for complex interactive behaviors in robotics.

Findings

Successfully learned diverse motor skills including obstacle navigation and object manipulation.

Policies exhibit consistent motion styles and reduced joint torques.

Transferred to real robot without additional fine-tuning.

Abstract

Real-world autonomous missions often require rich interaction with nearby objects, such as doors or switches, along with effective navigation. However, such complex behaviors are difficult to learn because they involve both high-level planning and low-level motor control. We present a novel framework, Cascaded Compositional Residual Learning (CCRL), which learns composite skills by recursively leveraging a library of previously learned control policies. Our framework learns multiplicative policy composition, task-specific residual actions, and synthetic goal information simultaneously while freezing the prerequisite policies. We further explicitly control the style of the motion by regularizing residual actions. We show that our framework learns joint-level control policies for a diverse set of motor skills ranging from basic locomotion to complex interactive navigation, including navigating around obstacles, pushing objects, crawling under a table, pushing a door open with its leg, and holding it open while walking through it. The proposed CCRL framework leads to policies with consistent styles and lower joint torques, which we successfully transfer to a real Unitree A1 robot without any additional fine-tuning.