KERV: Kinematic-Rectified Speculative Decoding for Embodied VLA Models

TL;DR

KERV introduces a kinematic-rectified speculative decoding framework that enhances the inference speed of vision-language-action models by integrating robotic kinematics, reducing computational costs while maintaining high success rates.

Contribution

The paper proposes a novel KERV framework combining kinematic prediction with speculative decoding to improve speed and accuracy in embodied VLA models.

Findings

Achieves 27% to 37% acceleration in inference speed.

Maintains nearly no loss in success rate across tasks.

Effectively reduces the need for costly re-inference.

Abstract

Vision-Language-Action (VLA) models build a token-domain robot control paradigm, yet suffer from low speed. Speculative Decoding (SD) is an optimization strategy that can boost inference speed. Two key issues emerge when integrating VLA and SD: first, SD relies on re-inference to address token errors, which is computationally expensive; second, to mitigate token errors, the acceptance threshold in SD requires careful adjustment. Existing works fail to address the above two issues effectively. Meanwhile, as the bridge between AI and the physical world, existing embodied intelligence has overlooked the application of robotic kinematics. To address these issues, we innovatively combine token-domain VLA models with kinematic-domain prediction for SD, proposing a kinematic-rectified SD framework named KERV. We employ a kinematics-based Kalman Filter to predict actions and compensate for SD errors, avoiding costly re-inference. Moreover, we design a kinematics-based adjustment strategy to dynamically rectify the acceptance threshold, addressing the difficulty of threshold determination. Experimental results across diverse tasks and environments demonstrate that KERV achieves 27%~37% acceleration with nearly no Success Rate loss.