Real-Time Iteration Scheme for Diffusion Policy

TL;DR

This paper introduces a real-time iteration scheme for diffusion policies in robotic manipulation, significantly reducing inference time without retraining, enabling efficient use of large pre-trained models in latency-critical tasks.

Contribution

It adapts the Real-Time Iteration scheme from optimal control to diffusion inference, providing a resource-efficient method to accelerate robotic policies without additional training or policy redesign.

Findings

Substantial reduction in inference time demonstrated in simulations

Maintains comparable performance to full-step denoising methods

Effective handling of discrete actions like grasping in manipulation tasks

Abstract

Diffusion Policies have demonstrated impressive performance in robotic manipulation tasks. However, their long inference time, resulting from an extensive iterative denoising process, and the need to execute an action chunk before the next prediction to maintain consistent actions limit their applicability to latency-critical tasks or simple tasks with a short cycle time. While recent methods explored distillation or alternative policy structures to accelerate inference, these often demand additional training, which can be resource-intensive for large robotic models. In this paper, we introduce a novel approach inspired by the Real-Time Iteration (RTI) Scheme, a method from optimal control that accelerates optimization by leveraging solutions from previous time steps as initial guesses for subsequent iterations. We explore the application of this scheme in diffusion inference and propose a scaling-based method to effectively handle discrete actions, such as grasping, in robotic manipulation. The proposed scheme significantly reduces runtime computational costs without the need for distillation or policy redesign. This enables a seamless integration into many pre-trained diffusion-based models, in particular, to resource-demanding large models. We also provide theoretical conditions for the contractivity which could be useful for estimating the initial denoising step. Quantitative results from extensive simulation experiments show a substantial reduction in inference time, with comparable overall performance compared with Diffusion Policy using full-step denoising. Our project page with additional resources is available at: https://rti-dp.github.io/.