Continual Policy Distillation of Reinforcement Learning-based Controllers for Soft Robotic In-Hand Manipulation

TL;DR

This paper presents a Continual Policy Distillation framework that enables soft robotic hands to learn versatile, adaptive in-hand manipulation skills across different objects by transferring knowledge from multiple expert policies while mitigating forgetting.

Contribution

The novel CPD framework combines policy distillation with exemplar-based rehearsal to improve adaptability and generalization in soft robotic in-hand manipulation tasks.

Findings

Effective knowledge transfer from multiple experts

Enhanced generalization and adaptability in manipulation

Mitigated catastrophic forgetting during learning

Abstract

Dexterous manipulation, often facilitated by multi-fingered robotic hands, holds solid impact for real-world applications. Soft robotic hands, due to their compliant nature, offer flexibility and adaptability during object grasping and manipulation. Yet, benefits come with challenges, particularly in the control development for finger coordination. Reinforcement Learning (RL) can be employed to train object-specific in-hand manipulation policies, but limiting adaptability and generalizability. We introduce a Continual Policy Distillation (CPD) framework to acquire a versatile controller for in-hand manipulation, to rotate different objects in shape and size within a four-fingered soft gripper. The framework leverages Policy Distillation (PD) to transfer knowledge from expert policies to a continually evolving student policy network. Exemplar-based rehearsal methods are then integrated to mitigate catastrophic forgetting and enhance generalization. The performance of the CPD framework over various replay strategies demonstrates its effectiveness in consolidating knowledge from multiple experts and achieving versatile and adaptive behaviours for in-hand manipulation tasks.