Refining Compositional Diffusion for Reliable Long-Horizon Planning

TL;DR

Refining Compositional Diffusion (RCD) is a guidance technique that improves long-horizon planning by steering diffusion models toward globally coherent plans, addressing mode-averaging issues in multimodal local plan distributions.

Contribution

RCD introduces a training-free guidance method using self-reconstruction error and overlap consistency to enhance compositional diffusion for reliable long-horizon planning.

Findings

RCD outperforms existing methods on challenging long-horizon tasks.

It effectively mitigates mode-averaging in multimodal plan distributions.

Experiments include locomotion, object manipulation, and pixel-based tasks.

Abstract

Compositional diffusion planning generates long-horizon trajectories by stitching together overlapping short-horizon segments through score composition. However, when local plan distributions are multimodal, existing compositional methods suffer from mode-averaging, where averaging incompatible local modes leads to plans that are neither locally feasible nor globally coherent. We propose Refining Compositional Diffusion (RCD), a training-free guidance method that steers compositional sampling toward high-density, globally coherent plans. RCD leverages the self-reconstruction error of a pretrained diffusion model as a proxy for the log-density of composed plans, combined with an overlap consistency term that enforces consistency at segment boundaries. We show that the combined guidance concentrates sampling on high-density plans that mitigate mode-averaging. Experiments on challenging long-horizon tasks from OGBench, including locomotion, object manipulation, and pixel-based observations, demonstrate that RCD consistently outperforms existing methods.