Point Bridge: 3D Representations for Cross Domain Policy Learning

TL;DR

Point Bridge introduces a domain-agnostic point-based representation framework that enables zero-shot sim-to-real transfer of robotic manipulation policies trained solely on synthetic data, significantly reducing the need for real-world datasets.

Contribution

The paper presents a novel point-based representation approach combined with vision-language models and transformer policies for effective zero-shot sim-to-real transfer without explicit visual alignment.

Findings

Up to 44% improvement in zero-shot transfer performance.

Up to 66% gains with limited real data.

Outperforms prior vision-based co-training methods.

Abstract

Robot foundation models are beginning to deliver on the promise of generalist robotic agents, yet progress remains constrained by the scarcity of large-scale real-world manipulation datasets. Simulation and synthetic data generation offer a scalable alternative, but their usefulness is limited by the visual domain gap between simulation and reality. In this work, we present Point Bridge, a framework that leverages unified, domain-agnostic point-based representations to unlock synthetic datasets for zero-shot sim-to-real policy transfer, without explicit visual or object-level alignment. Point Bridge combines automated point-based representation extraction via Vision-Language Models (VLMs), transformer-based policy learning, and efficient inference-time pipelines to train capable real-world manipulation agents using only synthetic data. With additional co-training on small sets of real demonstrations, Point Bridge further improves performance, substantially outperforming prior vision-based sim-and-real co-training methods. It achieves up to 44% gains in zero-shot sim-to-real transfer and up to 66% with limited real data across both single-task and multitask settings. Videos of the robot are best viewed at: https://pointbridge3d.github.io/