# Collapse and collision aware grasping for cluttered shelf picking

## TL;DR

This paper introduces a robotic system that uses physics simulations to safely and efficiently pick items from cluttered shelves.

## Contribution

The novel contribution is a physics-aware grasp planner that predicts and avoids collisions and collapses during shelf picking.

## Key findings

- The physics-based approach outperforms heuristic methods in both structured and unstructured box stacks.
- The system improves retrieval efficiency and success rates in real-world experiments.

## Abstract

In modern smart factories, automated shelf picking must deliver high throughput, flexibility, and safe human–robot collaboration. In these environments, efficient and safe retrieval of stacked objects is a significant challenge due to complex spatial dependencies and structural inter-dependencies. Traditional vision-based methods excel at object localization but often lack the physical reasoning required to predict the consequences of extraction, leading to unintended collisions and collapses. This paper proposes a collapse and collision-aware grasp planner that integrates dynamic physics simulations for robotic decision-making. Using a single image and depth map, an approximate 3D representation of the scene is reconstructed in a simulation environment, enabling the robot to evaluate different retrieval strategies before execution. Two approaches: 1) heuristic-based and 2) physics-based are proposed for both single-box extraction and shelf clearance tasks. Extensive real-world experiments on structured and unstructured box stacks, along with validation using datasets from existing databases, show that our physics-aware method significantly improves efficiency and success rates compared to baseline heuristics. A video demonstrating the real-world implementation of our proposed system is available at: https://youtu.be/GBWMiNIHUlU.