Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning

TL;DR

This paper introduces DMPEL, a dynamic expert library framework that enhances lifelong robot learning by efficiently sharing knowledge, reducing forgetting, and enabling flexible task adaptation without relying on task identifiers.

Contribution

The paper proposes DMPEL, a novel lifelong learning method that builds a progressive expert library with a lightweight router, improving transfer and reducing forgetting in robot learning.

Findings

Outperforms state-of-the-art methods on LIBERO benchmark

Uses minimal trainable parameters and storage

Achieves higher success rates during continual adaptation

Abstract

A generalist agent must continuously learn and adapt throughout its lifetime, achieving efficient forward transfer while minimizing catastrophic forgetting. Previous work within the dominant pretrain-then-finetune paradigm has explored parameter-efficient fine-tuning for single-task adaptation, effectively steering a frozen pretrained model with a small number of parameters. However, in the context of lifelong learning, these methods rely on the impractical assumption of a test-time task identifier and restrict knowledge sharing among isolated adapters. To address these limitations, we propose Dynamic Mixture of Progressive Parameter-Efficient Expert Library (DMPEL) for lifelong robot learning. DMPEL progressively builds a low-rank expert library and employs a lightweight router to dynamically combine experts into an end-to-end policy, enabling flexible and efficient lifelong forward transfer. Furthermore, by leveraging the modular structure of the fine-tuned parameters, we introduce expert coefficient replay, which guides the router to accurately retrieve frozen experts for previously encountered tasks. This technique mitigates forgetting while being significantly more storage- and computation-efficient than experience replay over the entire policy. Extensive experiments on the lifelong robot learning benchmark LIBERO demonstrate that our framework outperforms state-of-the-art lifelong learning methods in success rates during continual adaptation, while utilizing minimal trainable parameters and storage.