Local Manifold Approximation and Projection for Manifold-Aware Diffusion Planning

TL;DR

This paper introduces LoMAP, a training-free method that projects guided samples onto a low-rank subspace to improve the reliability of diffusion-based planning in long-horizon, offline reinforcement learning tasks.

Contribution

We propose LoMAP, a novel manifold projection technique that enhances diffusion planning by reducing infeasible trajectories without requiring additional training.

Findings

LoMAP effectively prevents infeasible trajectory generation.

Incorporating LoMAP improves performance on offline RL benchmarks.

LoMAP enhances hierarchical diffusion planning with additional performance gains.

Abstract

Recent advances in diffusion-based generative modeling have demonstrated significant promise in tackling long-horizon, sparse-reward tasks by leveraging offline datasets. While these approaches have achieved promising results, their reliability remains inconsistent due to the inherent stochastic risk of producing infeasible trajectories, limiting their applicability in safety-critical applications. We identify that the primary cause of these failures is inaccurate guidance during the sampling procedure, and demonstrate the existence of manifold deviation by deriving a lower bound on the guidance gap. To address this challenge, we propose Local Manifold Approximation and Projection (LoMAP), a training-free method that projects the guided sample onto a low-rank subspace approximated from offline datasets, preventing infeasible trajectory generation. We validate our approach on standard offline reinforcement learning benchmarks that involve challenging long-horizon planning. Furthermore, we show that, as a standalone module, LoMAP can be incorporated into the hierarchical diffusion planner, providing further performance enhancements.