Adaptive Online Replanning with Diffusion Models

TL;DR

This paper introduces a method for effective replanning with diffusion models in robotic control, improving task success rates and handling stochastic environments by determining optimal replanning times and trajectory adjustments.

Contribution

It proposes a principled approach to decide when to replan based on likelihood estimates and a method to replan trajectories to maintain goal consistency, enhancing diffusion-based planning.

Findings

38% performance improvement over previous diffusion planners

Effective handling of stochastic and long-horizon tasks

Improved replanning strategy increases success rates

Abstract

Diffusion models have risen as a promising approach to data-driven planning, and have demonstrated impressive robotic control, reinforcement learning, and video planning performance. Given an effective planner, an important question to consider is replanning -- when given plans should be regenerated due to both action execution error and external environment changes. Direct plan execution, without replanning, is problematic as errors from individual actions rapidly accumulate and environments are partially observable and stochastic. Simultaneously, replanning at each timestep incurs a substantial computational cost, and may prevent successful task execution, as different generated plans prevent consistent progress to any particular goal. In this paper, we explore how we may effectively replan with diffusion models. We propose a principled approach to determine when to replan, based on the diffusion model's estimated likelihood of existing generated plans. We further present an approach to replan existing trajectories to ensure that new plans follow the same goal state as the original trajectory, which may efficiently bootstrap off previously generated plans. We illustrate how a combination of our proposed additions significantly improves the performance of diffusion planners leading to 38\% gains over past diffusion planning approaches on Maze2D, and further enables the handling of stochastic and long-horizon robotic control tasks. Videos can be found on the anonymous website: \url{https://vis-www.cs.umass.edu/replandiffuser/}.