Diffusion Meets Options: Hierarchical Generative Skill Composition for Temporally-Extended Tasks

TL;DR

This paper introduces DOPPLER, a hierarchical framework combining diffusion models and reinforcement learning to generate and update long-horizon, temporally-extended plans for robots based on linear temporal logic instructions.

Contribution

It presents a novel hierarchical approach that integrates diffusion models with reinforcement learning for efficient, instruction-driven trajectory planning in robotics.

Findings

DOPPLER effectively generates trajectories satisfying complex temporal objectives.

The method improves speed and diversity of options through determinantal-guided sampling.

Experiments show successful navigation and manipulation tasks with obstacle avoidance and visitation sequences.

Abstract

Safe and successful deployment of robots requires not only the ability to generate complex plans but also the capacity to frequently replan and correct execution errors. This paper addresses the challenge of long-horizon trajectory planning under temporally extended objectives in a receding horizon manner. To this end, we propose DOPPLER, a data-driven hierarchical framework that generates and updates plans based on instruction specified by linear temporal logic (LTL). Our method decomposes temporal tasks into chain of options with hierarchical reinforcement learning from offline non-expert datasets. It leverages diffusion models to generate options with low-level actions. We devise a determinantal-guided posterior sampling technique during batch generation, which improves the speed and diversity of diffusion generated options, leading to more efficient querying. Experiments on robot navigation and manipulation tasks demonstrate that DOPPLER can generate sequences of trajectories that progressively satisfy the specified formulae for obstacle avoidance and sequential visitation. Demonstration videos are available online at: https://philiptheother.github.io/doppler/.