CRL-VLA: Continual Vision-Language-Action Learning

TL;DR

CRL-VLA introduces a continual learning framework for vision-language-action models that balances stability and plasticity, enabling lifelong robotic manipulation with theoretical guarantees and superior performance on benchmarks.

Contribution

It presents a novel continual reinforcement learning method with a dual-critic architecture and theoretical bounds, improving lifelong VLA model adaptation and retention.

Findings

Outperforms baselines on LIBERO benchmark

Effectively balances stability and plasticity

Provides theoretical performance bounds

Abstract

Lifelong learning is critical for embodied agents in open-world environments, where reinforcement learning fine-tuning has emerged as an important paradigm to enable Vision-Language-Action (VLA) models to master dexterous manipulation through environmental interaction. Thus, Continual Reinforcement Learning (CRL) is a promising pathway for deploying VLA models in lifelong robotic scenarios, yet balancing stability (retaining old skills) and plasticity (learning new ones) remains a formidable challenge for existing methods. We introduce CRL-VLA, a framework for continual post-training of VLA models with rigorous theoretical bounds. We derive a unified performance bound linking the stability-plasticity trade-off to goal-conditioned advantage magnitude, scaled by policy divergence. CRL-VLA resolves this dilemma via asymmetric regulation: constraining advantage magnitudes on prior tasks while enabling controlled growth on new tasks. This is realized through a simple but effective dual-critic architecture with novel Goal-Conditioned Value Formulation (GCVF), where a frozen critic anchors semantic consistency and a trainable estimator drives adaptation. Experiments on the LIBERO benchmark demonstrate that CRL-VLA effectively harmonizes these conflicting objectives, outperforming baselines in both anti-forgetting and forward adaptation.