Training-Time Action Conditioning for Efficient Real-Time Chunking

TL;DR

This paper introduces a training-time action conditioning method for real-time chunking in vision-language-action models, reducing inference latency and computational overhead while maintaining task performance in robot control tasks.

Contribution

The authors propose a training-time simulation of inference delay and conditioning on action prefixes, eliminating the need for inference-time inpainting in real-time robot control.

Findings

Training-time RTC outperforms inference-time RTC at higher delays.

Maintains task performance and speed in real-world experiments.

Reduces computational overhead without modifying model architecture.

Abstract

Real-time chunking (RTC) enables vision-language-action models (VLAs) to generate smooth, reactive robot trajectories by asynchronously predicting action chunks and conditioning on previously committed actions via inference-time inpainting. However, this inpainting method introduces computational overhead that increases inference latency. In this work, we propose a simple alternative: simulating inference delay at training time and conditioning on action prefixes directly, eliminating any inference-time overhead. Our method requires no modifications to the model architecture or robot runtime, and can be implemented with only a few additional lines of code. In simulated experiments, we find that training-time RTC outperforms inference-time RTC at higher inference delays. In real-world experiments on box building and espresso making tasks with the $\pi_{0.6}$ VLA, we demonstrate that training-time RTC maintains both task performance and speed parity with inference-time RTC while being computationally cheaper. Our results suggest that training-time action conditioning is a practical drop-in replacement for inference-time inpainting in real-time robot control.