Fast ECoT: Efficient Embodied Chain-of-Thought via Thoughts Reuse

TL;DR

Fast ECoT significantly reduces inference latency in embodied reasoning models by caching, parallelising, and asynchronously scheduling reasoning steps, enabling more practical real-time vision-language-action applications without retraining.

Contribution

It introduces a novel inference acceleration method for ECoT that leverages reasoning structure, enabling real-time deployment without model modifications or additional training.

Findings

Up to 7.5% latency reduction in experiments

Maintains or improves task success rate

Enhances reasoning faithfulness

Abstract

Embodied Chain-of-Thought (ECoT) reasoning enhances vision-language-action (VLA) models by improving performance and interpretability through intermediate reasoning steps. However, its sequential autoregressive token generation introduces significant inference latency, limiting real-time deployment. We propose Fast ECoT, an inference-time acceleration method that exploits the structured and repetitive nature of ECoT to (1) cache and reuse high-level reasoning across timesteps and (2) parallelise the generation of modular reasoning steps. Additionally, we introduce an asynchronous scheduler that decouples reasoning from action decoding, further boosting responsiveness. Fast ECoT requires no model changes or additional training and integrates easily into existing VLA pipelines. Experiments in both simulation (LIBERO) and real-world robot tasks show up to a 7.5% reduction in latency with comparable or improved task success rate and reasoning faithfulness, bringing ECoT policies closer to practical real-time deployment.