Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling

TL;DR

This paper introduces Bidirectional Decoding, a test-time inference method that enhances action chunking in robot learning by balancing temporal consistency and reactivity, improving policy performance in various tasks.

Contribution

The paper proposes Bidirectional Decoding, a novel test-time sampling algorithm that improves action chunking by integrating backward and forward criteria for better policy adaptation.

Findings

BID improves performance across multiple benchmarks.

Action chunking captures temporal dependencies but reduces reactivity.

BID balances long-term consistency with short-term reactivity.

Abstract

Predicting and executing a sequence of actions without intermediate replanning, known as action chunking, is increasingly used in robot learning from human demonstrations. Yet, its effects on the learned policy remain inconsistent: some studies find it crucial for achieving strong results, while others observe decreased performance. In this paper, we first dissect how action chunking impacts the divergence between a learner and a demonstrator. We find that action chunking allows the learner to better capture the temporal dependencies in demonstrations but at the cost of reduced reactivity to unexpected states. To address this tradeoff, we propose Bidirectional Decoding (BID), a test-time inference algorithm that bridges action chunking with closed-loop adaptation. At each timestep, BID samples multiple candidate predictions and searches for the optimal one based on two criteria: (i) backward coherence, which favors samples that align with previous decisions; (ii) forward contrast, which seeks samples of high likelihood for future plans. By coupling decisions within and across action chunks, BID promotes both long-term consistency and short-term reactivity. Experimental results show that our method boosts the performance of two state-of-the-art generative policies across seven simulation benchmarks and two real-world tasks. Code and videos are available at https://bid-robot.github.io.