Coarse-to-Fine 3D Keyframe Transporter
Xupeng Zhu, David Klee, Dian Wang, Boce Hu, Haojie Huang, Arsh Tangri,, Robin Walters, Robert Platt

TL;DR
This paper introduces a bi-equivariant symmetry-aware keyframe transporter that improves manipulation task generalization and efficiency by leveraging symmetries and a coarse-to-fine SE(3) evaluation scheme, outperforming existing methods.
Contribution
It analyzes bi-equivariance in keyframe IL and proposes a novel, symmetry-aware Keyframe Transporter with a coarse-to-fine SE(3) reasoning scheme, enhancing generalization and efficiency.
Findings
Outperforms keyframe IL baselines by >10% in simulation
Achieves 55% improvement in physical experiments
Introduces a symmetry-aware, coarse-to-fine action evaluation method
Abstract
Recent advances in Keyframe Imitation Learning (IL) have enabled learning-based agents to solve a diverse range of manipulation tasks. However, most approaches ignore the rich symmetries in the problem setting and, as a consequence, are sample-inefficient. This work identifies and utilizes the bi-equivariant symmetry within Keyframe IL to design a policy that generalizes to transformations of both the workspace and the objects grasped by the gripper. We make two main contributions: First, we analyze the bi-equivariance properties of the keyframe action scheme and propose a Keyframe Transporter derived from the Transporter Networks, which evaluates actions using cross-correlation between the features of the grasped object and the features of the scene. Second, we propose a computationally efficient coarse-to-fine SE(3) action evaluation scheme for reasoning the intertwined translation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
Topics3D IC and TSV technologies · Modular Robots and Swarm Intelligence
