Real-Time Robot Execution with Masked Action Chunking

TL;DR

This paper introduces REMAC, a method for improving real-time robot execution by learning corrective adjustments to handle intra-chunk inconsistencies during asynchronous inference, resulting in more reliable and faster task performance.

Contribution

We propose REMAC, a novel approach that learns corrective adjustments for action chunks to enhance robustness in asynchronous robot inference, addressing intra-chunk inconsistency.

Findings

REMAC improves task completion rates in real-world robot experiments.

The method maintains robustness under varying delays.

It enables faster and more reliable robot execution.

Abstract

Real-time execution is essential for cyber-physical systems such as robots. These systems operate in dynamic real-world environments where even small delays can undermine responsiveness and compromise performance. Asynchronous inference has recently emerged as a system-level paradigm for real-time robot manipulation, enabling the next action chunk to be predicted while the current one is being executed. While this approach achieves real-time responsiveness, naive integration often results in execution failure. Previous methods attributed this failure to inter-chunk discontinuity and developed test-time algorithms to smooth chunk boundaries. In contrast, we identify another critical yet overlooked factor: intra-chunk inconsistency, where the robot's executed action chunk partially misaligns with its current perception. To address this, we propose REMAC, which learns corrective adjustments on the pretrained policy through masked action chunking, enabling the policy to remain resilient under mismatches between intended actions and actual execution during asynchronous inference. In addition, we introduce a prefix-preserved sampling procedure to reinforce inter-chunk continuity. Overall, our method delivers more reliable policies without incurring additional latency. Extensive experiments in both simulation and real-world settings demonstrate that our method enables faster task execution, maintains robustness across varying delays, and consistently achieves higher completion rates.