Temporal Action Selection for Action Chunking

TL;DR

This paper introduces TAS, a novel algorithm for action chunking that balances reactivity, decision consistency, and motion coherence, significantly improving success rates and training efficiency in robotic tasks.

Contribution

The paper proposes TAS, a new method that dynamically selects action chunks to enhance reactivity and consistency in action chunking for Learning from Demonstration.

Findings

TAS improves success rates by up to 73.3%.

Integrating TAS with residual RL enhances training efficiency.

Experiments confirm TAS's effectiveness in simulation and real robots.

Abstract

Action chunking is a widely adopted approach in Learning from Demonstration (LfD). By modeling multi-step action chunks rather than single-step actions, action chunking significantly enhances modeling capabilities for human expert policies. However, the reduced decision frequency restricts the utilization of recent observations, degrading reactivity - particularly evident in the inadequate adaptation to sensor noise and dynamic environmental changes. Existing efforts to address this issue have primarily resorted to trading off reactivity against decision consistency, without achieving both. To address this limitation, we propose a novel algorithm, Temporal Action Selector (TAS), which caches predicted action chunks from multiple timesteps and dynamically selects the optimal action through a lightweight selector network. TAS achieves balanced optimization across three critical dimensions: reactivity, decision consistency, and motion coherence. Experiments across multiple tasks with diverse base policies show that TAS significantly improves success rates - yielding an absolute gain of up to 73.3%. Furthermore, integrating TAS as a base policy with residual reinforcement learning (RL) substantially enhances training efficiency and elevates the performance plateau. Experiments in both simulation and physical robots confirm the method's efficacy.