TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformers

TL;DR

TwinBrainVLA introduces a dual-path vision-language-action model that preserves pre-trained knowledge while fine-tuning for robotic tasks, significantly improving manipulation performance.

Contribution

It proposes a novel asymmetric mixture-of-transformers architecture with dual VLM pathways to retain general knowledge during robotic fine-tuning.

Findings

Achieves superior performance on SimplerEnv and RoboCasa benchmarks.

Effectively mitigates catastrophic forgetting in robotic fine-tuning.

Enhances complex manipulation task success rates.

Abstract

The fundamental premise of Vision-Language-Action (VLA) models is to harness the extensive general capabilities of pre-trained Vision-Language Models (VLMs) for generalized embodied intelligence. However, standard robotic fine-tuning inevitably disrupts the pre-trained feature space, leading to "catastrophic forgetting" that compromises the general visual understanding we aim to leverage. To effectively utilize the uncorrupted general capabilities of VLMs for robotic tasks, we propose TwinBrainVLA, which coordinates two isomorphic VLM pathways: a frozen generalist (also called "Left Brain") and a trainable specialist (also called "Right Brain"). Our architecture utilizes a Asymmetric Mixture-of-Transformers (AsyMoT) mechanism, enabling the Right Brain to dynamically query and fuse intact semantic knowledge from the Left Brain with proprioceptive states. This fused representation conditions a flow-matching action expert for precise continuous control. Empirical results on SimplerEnv and RoboCasa benchmarks demonstrate that by explicitly retaining general capabilities, TwinBrainVLA achieves substantial performance gains over baseline models in complex manipulation tasks.