TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models

TL;DR

This paper introduces TAIL, a framework that uses parameter-efficient fine-tuning techniques like LoRA to adapt large pretrained models for new control tasks in robotics, achieving high performance with minimal data and computational resources.

Contribution

The paper proposes TAIL, a novel approach that applies parameter-efficient fine-tuning methods to large pretrained models for imitation learning in control domains, enabling data-efficient and continual adaptation.

Findings

LoRA-based TAIL achieves superior performance with only 1% of trainable parameters.

TAIL prevents catastrophic forgetting during continual learning.

Parameter-efficient fine-tuning outperforms full fine-tuning in control tasks.

Abstract

The full potential of large pretrained models remains largely untapped in control domains like robotics. This is mainly because of the scarcity of data and the computational challenges associated with training or fine-tuning these large models for such applications. Prior work mainly emphasizes either effective pretraining of large models for decision-making or single-task adaptation. But real-world problems will require data-efficient, continual adaptation for new control tasks. Recognizing these constraints, we introduce TAIL (Task-specific Adapters for Imitation Learning), a framework for efficient adaptation to new control tasks. Inspired by recent advancements in parameter-efficient fine-tuning in language domains, we explore efficient fine-tuning techniques -- e.g., Bottleneck Adapters, P-Tuning, and Low-Rank Adaptation (LoRA) -- in TAIL to adapt large pretrained models for new tasks with limited demonstration data. Our extensive experiments in large-scale language-conditioned manipulation tasks comparing prevalent parameter-efficient fine-tuning techniques and adaptation baselines suggest that TAIL with LoRA can achieve the best post-adaptation performance with only 1\% of the trainable parameters of full fine-tuning, while avoiding catastrophic forgetting and preserving adaptation plasticity in continual learning settings.