GraspView: Active Perception Scoring and Best-View Optimization for Robotic Grasping in Cluttered Environments

TL;DR

GraspView is an RGB-only robotic grasping system that uses active perception and multi-view reconstruction to improve grasp success in cluttered, occluded, and transparent object environments without relying on depth sensors.

Contribution

It introduces a novel RGB-only pipeline with global scene reconstruction, active view selection, and scale calibration, enabling robust grasping in challenging cluttered scenes.

Findings

Outperforms RGB-D and single-view baselines in cluttered environments.

Effectively handles occlusion and transparent objects.

Achieves reliable grasping without depth sensors.

Abstract

Robotic grasping is a fundamental capability for autonomous manipulation, yet remains highly challenging in cluttered environments where occlusion, poor perception quality, and inconsistent 3D reconstructions often lead to unstable or failed grasps. Conventional pipelines have widely relied on RGB-D cameras to provide geometric information, which fail on transparent or glossy objects and degrade at close range. We present GraspView, an RGB-only robotic grasping pipeline that achieves accurate manipulation in cluttered environments without depth sensors. Our framework integrates three key components: (i) global perception scene reconstruction, which provides locally consistent, up-to-scale geometry from a single RGB view and fuses multi-view projections into a coherent global 3D scene; (ii) a render-and-score active perception strategy, which dynamically selects next-best-views to reveal occluded regions; and (iii) an online metric alignment module that calibrates VGGT predictions against robot kinematics to ensure physical scale consistency. Building on these tailor-designed modules, GraspView performs best-view global grasping, fusing multi-view reconstructions and leveraging GraspNet for robust execution. Experiments on diverse tabletop objects demonstrate that GraspView significantly outperforms both RGB-D and single-view RGB baselines, especially under heavy occlusion, near-field sensing, and with transparent objects. These results highlight GraspView as a practical and versatile alternative to RGB-D pipelines, enabling reliable grasping in unstructured real-world environments.