CapVector: Learning Transferable Capability Vectors in Parametric Space for Vision-Language-Action Models

TL;DR

This paper introduces CapVector, a method to learn transferable capability vectors in parametric space for vision-language-action models, enhancing performance and adaptability with reduced computational costs.

Contribution

It decouples auxiliary training objectives in parameter space, enabling efficient transfer of capabilities and improved generalization in vision-language-action models.

Findings

Capability vectors are effective across diverse models.

Merged models perform comparably to auxiliary finetuned baselines.

Vectors generalize well to new environments and embodiments.

Abstract

This paper proposes a novel approach to address the challenge that pretrained VLA models often fail to effectively improve performance and reduce adaptation costs during standard supervised finetuning (SFT). Some advanced finetuning methods with auxiliary training objectives can improve performance and reduce the number of convergence steps. However, they typically incur significant computational overhead due to the additional losses from auxiliary objectives. To simultaneously achieve the enhanced capabilities of auxiliary training with the simplicity of standard SFT, we decouple the two objectives of auxiliary-objective SFT within the parameter space, namely, enhancing general capabilities and fitting task-specific action distributions. To deliver the goal, we only need to train the model to converge on a small-scale task set using two distinct training strategies, resulting in two finetuned models. The parameters' difference between the two models can then be interpreted as capability vectors provided by auxiliary objectives. These vectors are then merged with pretrained parameters to form a capability-enhanced meta model. Moreover, when standard SFT is augmented with a lightweight orthogonal regularization loss, the merged model attains performance comparable to auxiliary finetuned baselines with reduced computational overhead. Internal and external experiments demonstrate that our capability vectors (1) are effective and versatile across diverse models, (2) can generalize to novel environments and embodiments out of the box.