Health-Conditioned Vision-Language-Action Models for Malfunction-Aware Robot Control

TL;DR

This paper introduces a health-conditioned vision-language-action model for robots that adapts to physical failures by incorporating a health vector, improving task success with degraded joints.

Contribution

The authors propose a novel health-aware VLA model with a health projector module, trained on malfunction data, enabling robots to adapt to joint degradations.

Findings

Model successfully operates with various degraded joint configurations.

Lightweight addition enables adaptation without retraining the entire model.

Code and dataset will be publicly available at the provided GitHub link.

Abstract

Research on Vision Language Action (VLA) models has been increasing rapidly in recent years. Although some of them focus on detecting, preventing, and recovering from task failures, they usually don't deal with adapting to robot's physical failures. In real-life scenarios, most robots face physical degradations in various ways such as joint degradation, actuator failure, or weak gripper. We introduce malfunction-aware (health-conditioned) VLA that takes a health vector as an input that gives information about robots' joints' operation angle and torque capability, and adapts its predictions to complete the tasks with the degraded joints. To achieve this, we inject a Health Projector module to the VLA-Adapter architecture and train it on malfunction robot data we collected on the LIBERO environment [1]. We collect 128 teleoperated episodes on Libero-Spatial tasks. Our results show that, with a very lightweight addition, the model can learn to operate successfully with different configurations of degraded joints which the default pretrained VLA-Adapter's Libero-Spatial-Pro model cannot. The code and dataset will be available soon at https://github.com/h-arslan/health-aware-vla