A Flexible Field-Based Policy Learning Framework for Diverse Robotic Systems and Sensors

TL;DR

This paper introduces a versatile visuomotor learning framework that combines diffusion policy control with 3D scene understanding, enabling robots with different sensors and configurations to learn manipulation tasks efficiently and generalize across platforms.

Contribution

It presents a modular, cross-robot framework integrating diffusion policies and 3D scene representations, supporting diverse sensors and robot configurations for manipulation learning.

Findings

Achieved 80% success rate after 100 demonstrations in a grasp and lift task.

Demonstrated robust skill transfer across different robot platforms and sensors.

Framework supports flexible data collection and real-time control for diverse robotic systems.

Abstract

We present a cross robot visuomotor learning framework that integrates diffusion policy based control with 3D semantic scene representations from D3Fields to enable category level generalization in manipulation. Its modular design supports diverse robot camera configurations including UR5 arms with Microsoft Azure Kinect arrays and bimanual manipulators with Intel RealSense sensors through a low latency control stack and intuitive teleoperation. A unified configuration layer enables seamless switching between setups for flexible data collection training and evaluation. In a grasp and lift block task the framework achieved an 80 percent success rate after only 100 demonstration episodes demonstrating robust skill transfer between platforms and sensing modalities. This design paves the way for scalable real world studies in cross robotic generalization.