KDPE: A Kernel Density Estimation Strategy for Diffusion Policy Trajectory Selection

TL;DR

KDPE introduces a kernel density estimation strategy to filter out unsafe trajectories generated by diffusion policies, improving robot behavior cloning by reducing outliers and maintaining computational efficiency.

Contribution

The paper proposes KDPE, a novel manifold-aware kernel density estimation method that enhances diffusion policy trajectories for safer and more reliable robot behavior cloning.

Findings

KDPE outperforms Diffusion Policy in simulated tasks.

KDPE reduces outlier trajectories during policy execution.

KDPE maintains low computational overhead at test time.

Abstract

Learning robot policies that capture multimodality in the training data has been a long-standing open challenge for behavior cloning. Recent approaches tackle the problem by modeling the conditional action distribution with generative models. One of these approaches is Diffusion Policy, which relies on a diffusion model to denoise random points into robot action trajectories. While achieving state-of-the-art performance, it has two main drawbacks that may lead the robot out of the data distribution during policy execution. First, the stochasticity of the denoising process can highly impact on the quality of generated trajectory of actions. Second, being a supervised learning approach, it can learn data outliers from the dataset used for training. Recent work focuses on mitigating these limitations by combining Diffusion Policy either with large-scale training or with classical behavior cloning algorithms. Instead, we propose KDPE, a Kernel Density Estimation-based strategy that filters out potentially harmful trajectories output of Diffusion Policy while keeping a low test-time computational overhead. For Kernel Density Estimation, we propose a manifold-aware kernel to model a probability density function for actions composed of end-effector Cartesian position, orientation, and gripper state. KDPE overall achieves better performance than Diffusion Policy on simulated single-arm tasks and real robot experiments. Additional material and code are available on our project page at https://hsp-iit.github.io/KDPE/.