CRAFT: Adapting VLA Models to Contact-rich Manipulation via Force-aware Curriculum Fine-tuning

TL;DR

This paper introduces CRAFT, a curriculum fine-tuning framework that enhances vision-language-action models for contact-rich robot manipulation by integrating force signals, leading to improved success and generalization in real-world tasks.

Contribution

CRAFT is the first to incorporate a force-aware curriculum fine-tuning approach with a variational information bottleneck for VLA models in contact-rich manipulation.

Findings

CRAFT improves task success rates in real-world experiments.

The framework generalizes to unseen objects and task variations.

It effectively adapts across different VLA architectures.

Abstract

Vision-Language-Action (VLA) models have shown a strong capability in enabling robots to execute general instructions, yet they struggle with contact-rich manipulation tasks, where success requires precise alignment, stable contact maintenance, and effective handling of deformable objects. A fundamental challenge arises from the imbalance between high-entropy vision and language inputs and low-entropy but critical force signals, which often leads to over-reliance on perception and unstable control. To address this, we introduce CRAFT, a force-aware curriculum fine-tuning framework that integrates a variational information bottleneck module to regulate vision and language embeddings during early training. This curriculum strategy encourages the model to prioritize force signals initially, before progressively restoring access to the full multimodal information. To enable force-aware learning, we further design a homologous leader-follower teleoperation system that collects synchronized vision, language, and force data across diverse contact-rich tasks. Real-world experiments demonstrate that CRAFT consistently improves task success, generalizes to unseen objects and novel task variations, and adapts effectively across diverse VLA architectures, enabling robust and generalizable contact-rich manipulation.