A Hierarchical Bin Packing Framework with Dual Manipulators via Heuristic Search and Deep Reinforcement Learning

TL;DR

This paper introduces a hierarchical framework combining deep reinforcement learning and heuristic search to optimize 2D bin packing with dual manipulators, effectively handling online, semi-online, and diverse practical scenarios.

Contribution

It presents a novel hierarchical approach integrating RL and heuristic search for 2D bin packing with dual manipulators, addressing online and semi-online challenges.

Findings

Achieves near-optimal bin utilization in various scenarios.

Handles diverse conditions including repacking and varying item information.

Demonstrates effective real-world performance in physics-based simulations.

Abstract

We address the bin packing problem (BPP), which aims to maximize bin utilization when packing a variety of items. The offline problem, where the complete information about the item set and their sizes is known in advance, is proven to be NP-hard. The semi-online and online variants are even more challenging, as full information about incoming items is unavailable. While existing methods have tackled both 2D and 3D BPPs, the 2D BPP remains underexplored in terms of fully maximizing utilization. We propose a hierarchical approach for solving the 2D online and semi-online BPP by combining deep reinforcement learning (RL) with heuristic search. The heuristic search selects which item to pack or unpack, determines the packing order, and chooses the orientation of each item, while the RL agent decides the precise position within the bin. Our method is capable of handling diverse scenarios, including repacking, varying levels of item information, differing numbers of accessible items, and coordination of dual manipulators. Experimental results demonstrate that our approach achieves near-optimal utilization across various practical scenarios, largely due to its repacking capability. In addition, the algorithm is evaluated in a physics-based simulation environment, where execution time is measured to assess its real-world performance.