Training Strategies for Efficient Embodied Reasoning

TL;DR

This paper investigates why robot chain-of-thought reasoning improves policy performance, identifies key mechanisms, and proposes lightweight alternatives that enhance efficiency and accuracy in vision-language-action models.

Contribution

It provides a mechanistic understanding of robot reasoning benefits and introduces simple, efficient reasoning strategies that outperform existing methods.

Findings

Reasoning improves representation learning and action prediction.

Attending to generated reasonings enhances policy performance.

Proposed methods achieve state-of-the-art results and faster inference.

Abstract

Robot chain-of-thought reasoning (CoT) -- wherein a model predicts helpful intermediate representations before choosing actions -- provides an effective method for improving the generalization and performance of robot policies, especially vision-language-action models (VLAs). While such approaches have been shown to improve performance and generalization, they suffer from core limitations, like needing specialized robot reasoning data and slow inference speeds. To design new robot reasoning approaches that address these issues, a more complete characterization of why reasoning helps policy performance is critical. We hypothesize several mechanisms by which robot reasoning improves policies -- (1) better representation learning, (2) improved learning curricularization, and (3) increased expressivity -- then devise simple variants of robot CoT reasoning to isolate and test each one. We find that learning to generate reasonings does lead to better VLA representations, while attending to the reasonings aids in actually leveraging these features for improved action prediction. Our results provide us with a better understanding of why CoT reasoning helps VLAs, which we use to introduce two simple and lightweight alternative recipes for robot reasoning. Our proposed approaches achieve significant performance gains over non-reasoning policies, state-of-the-art results on the LIBERO-90 benchmark, and a 3x inference speedup compared to standard robot reasoning.