FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulation

TL;DR

FD-VLA introduces a novel framework that integrates force awareness into vision-language-action models for contact-rich manipulation without physical force sensors, improving robustness and reducing hardware costs.

Contribution

The paper proposes a force distillation module that enables force-aware reasoning in VLA models without relying on physical force sensors, enhancing practicality and robustness.

Findings

Distilled force token outperforms direct force sensor measurements.

The approach reduces hardware costs by eliminating the need for physical force sensors.

Enhanced perception-action robustness in contact-rich tasks.

Abstract

Force sensing is a crucial modality for Vision-Language-Action (VLA) frameworks, as it enables fine-grained perception and dexterous manipulation in contact-rich tasks. We present Force-Distilled VLA (FD-VLA), a novel framework that integrates force awareness into contact-rich manipulation without relying on physical force sensors. The core of our approach is a Force Distillation Module (FDM), which distills force by mapping a learnable query token, conditioned on visual observations and robot states, into a predicted force token aligned with the latent representation of actual force signals. During inference, this distilled force token is injected into the pretrained VLM, enabling force-aware reasoning while preserving the integrity of its vision-language semantics. This design provides two key benefits: first, it allows practical deployment across a wide range of robots that lack expensive or fragile force-torque sensors, thereby reducing hardware cost and complexity; second, the FDM introduces an additional force-vision-state fusion prior to the VLM, which improves cross-modal alignment and enhances perception-action robustness in contact-rich scenarios. Surprisingly, our physical experiments show that the distilled force token outperforms direct sensor force measurements as well as other baselines, which highlights the effectiveness of this force-distilled VLA approach.