ISS Policy : Scalable Diffusion Policy with Implicit Scene Supervision

TL;DR

The paper introduces ISS Policy, a diffusion-based visuomotor policy leveraging implicit scene supervision to enhance robotic manipulation, achieving state-of-the-art results and strong real-world generalization.

Contribution

It proposes a novel implicit scene supervision module integrated with a diffusion policy for improved 3D scene understanding in robotic manipulation.

Findings

State-of-the-art performance on MetaWorld and Adroit tasks

Strong generalization and robustness in real-world experiments

Effective scaling with data and parameters

Abstract

Vision-based imitation learning has enabled impressive robotic manipulation skills, but its reliance on object appearance while ignoring the underlying 3D scene structure leads to low training efficiency and poor generalization. To address these challenges, we introduce \emph{Implicit Scene Supervision (ISS) Policy}, a 3D visuomotor DiT-based diffusion policy that predicts sequences of continuous actions from point cloud observations. We extend DiT with a novel implicit scene supervision module that encourages the model to produce outputs consistent with the scene's geometric evolution, thereby improving the performance and robustness of the policy. Notably, ISS Policy achieves state-of-the-art performance on both single-arm manipulation tasks (MetaWorld) and dexterous hand manipulation (Adroit). In real-world experiments, it also demonstrates strong generalization and robustness. Additional ablation studies show that our method scales effectively with both data and parameters. Code and videos will be released.