PPGuide: Steering Diffusion Policies with Performance Predictive Guidance

TL;DR

PPGuide is a lightweight, classifier-based framework that improves the robustness of diffusion policies in robotic manipulation by predicting and steering away from failure modes during inference.

Contribution

It introduces a novel self-supervised, attention-based learning method to guide diffusion policies away from failure modes in real-time.

Findings

Consistent performance improvements across multiple benchmarks.

Effective real-time guidance without extensive dataset augmentation.

Enhanced robustness in complex manipulation tasks.

Abstract

Diffusion policies have shown to be very efficient at learning complex, multi-modal behaviors for robotic manipulation. However, errors in generated action sequences can compound over time which can potentially lead to failure. Some approaches mitigate this by augmenting datasets with expert demonstrations or learning predictive world models which might be computationally expensive. We introduce Performance Predictive Guidance (PPGuide), a lightweight, classifier-based framework that steers a pre-trained diffusion policy away from failure modes at inference time. PPGuide makes use of a novel self-supervised process: it uses attention-based multiple instance learning to automatically estimate which observation-action chunks from the policy's rollouts are relevant to success or failure. We then train a performance predictor on this self-labeled data. During inference, this predictor provides a real-time gradient to guide the policy toward more robust actions. We validated our proposed PPGuide across a diverse set of tasks from the Robomimic and MimicGen benchmarks, demonstrating consistent improvements in performance.