Collaborate sim and real: Robot Bin Packing Learning in Real-world and Physical Engine

TL;DR

This paper introduces a hybrid reinforcement learning approach combining simulation and real-world data to improve the stability of robot bin packing, effectively bridging the simulation-to-reality gap and reducing packing failures.

Contribution

It proposes a collaborative framework that uses domain randomization and real-world fine-tuning to enhance RL agent generalization for stable bin packing in practical settings.

Findings

35% reduction in packing collapse in real-world deployments

Lower collapse rates in both simulated and real scenarios

Effective generalization across diverse physical parameters

Abstract

The 3D bin packing problem, with its diverse industrial applications, has garnered significant research attention in recent years. Existing approaches typically model it as a discrete and static process, while real-world applications involve continuous gravity-driven interactions. This idealized simplification leads to infeasible deployments (e.g., unstable packing) in practice. Simulations with physical engine offer an opportunity to emulate continuous gravity effects, enabling the training of reinforcement learning (RL) agents to address such limitations and improve packing stability. However, a simulation-to-reality gap persists due to dynamic variations in physical properties of real-world objects, such as various friction coefficients, elasticity, and non-uniform weight distributions. To bridge this gap, we propose a hybrid RL framework that collaborates with physical simulation with real-world data feedback. Firstly, domain randomization is applied during simulation to expose agents to a spectrum of physical parameters, enhancing their generalization capability. Secondly, the RL agent is fine-tuned with real-world deployment feedback, further reducing collapse rates. Extensive experiments demonstrate that our method achieves lower collapse rates in both simulated and real-world scenarios. Large-scale deployments in logistics systems validate the practical effectiveness, with a 35\% reduction in packing collapse compared to baseline methods.