StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perception

TL;DR

StereoPolicy enhances robotic manipulation by leveraging stereo image pairs to improve geometric reasoning without explicit 3D reconstruction, outperforming various baselines in simulation and real-world tests.

Contribution

Introduces StereoPolicy, a novel framework that uses stereo vision and a Stereo Transformer to improve visuomotor policies without requiring 3D reconstruction or calibration.

Findings

StereoPolicy outperforms RGB, RGB-D, point cloud, and multi-view baselines in simulation benchmarks.

StereoPolicy demonstrates effective real-robot manipulation in tabletop and bimanual settings.

Using stereo vision improves geometric reasoning and manipulation accuracy.

Abstract

Recent advances in robot imitation learning have yielded powerful visuomotor policies capable of manipulating a wide variety of objects directly from monocular visual inputs. However, monocular observations inherently lack reliable depth cues and spatial awareness, which are critical for precise manipulation in cluttered or geometrically complex scenes. To address this limitation, we introduce StereoPolicy, a new visuomotor policy learning framework that directly leverages synchronized stereo image pairs to strengthen geometric reasoning, without requiring explicit 3D reconstruction or camera calibration. StereoPolicy employs pretrained 2D vision encoders to process each image independently and fuses the resulting representations through a Stereo Transformer. This design implicitly captures spatial correspondence and disparity cues. The framework integrates seamlessly with diffusion-based and pretrained vision-language-action (VLA) policies, delivering consistent improvements over RGB, RGB-D, point cloud, and multi-view baselines across three simulation benchmarks: RoboMimic, RoboCasa, and OmniGibson. We further validate StereoPolicy on real-robot experiments spanning both tabletop and bimanual mobile manipulation settings. Our results underscore stereo vision as a scalable and robust modality that bridges 2D pretrained representations with 3D geometric understanding for robotic manipulation.