Mixed-Density Diffuser: Efficient Planning with Non-Uniform Temporal Resolution

TL;DR

The paper introduces Mixed-Density Diffuser, a diffusion planning method with tunable temporal densities, improving long-term planning efficiency and achieving state-of-the-art results on D4RL benchmarks.

Contribution

It proposes a novel diffusion planner with non-uniform, tunable temporal densities, outperforming existing methods in reinforcement learning tasks.

Findings

MDD surpasses SOTA Diffusion Veteran on multiple datasets.

MDD achieves a new SOTA on the D4RL benchmark.

Non-uniform density planning improves long-term dependency modeling.

Abstract

Recent studies demonstrate that diffusion planners benefit from sparse-step planning over single-step planning. Training models to skip steps in their trajectories helps capture long-term dependencies without additional memory or computational cost. However, predicting excessively sparse plans degrades performance. We hypothesize this temporal density threshold is non-uniform across a planning horizon and that certain parts of a predicted trajectory should be more densely generated. We propose Mixed-Density Diffuser (MDD), a diffusion planner where the densities throughout the horizon are tunable hyperparameters. We show that MDD surpasses the SOTA Diffusion Veteran (DV) framework across the Maze2D, Franka Kitchen, and Antmaze Datasets for Deep Data-Driven Reinforcement Learning (D4RL) task domains, achieving a new SOTA on the D4RL benchmark.