Learning Practically Feasible Policies for Online 3D Bin Packing

TL;DR

This paper presents a deep reinforcement learning approach to the online 3D bin packing problem, achieving high packing efficiency and practical feasibility for real-world robotic applications.

Contribution

It introduces a novel stacking tree analysis, decoupled dimension-wise policy learning, and a reward design to improve packing stability and collision avoidance in online 3D bin packing.

Findings

Outperforms state-of-the-art methods significantly

Achieves high packing stability and efficiency

Demonstrates practical usability in real-world scenarios

Abstract

We tackle the Online 3D Bin Packing Problem, a challenging yet practically useful variant of the classical Bin Packing Problem. In this problem, the items are delivered to the agent without informing the full sequence information. Agent must directly pack these items into the target bin stably without changing their arrival order, and no further adjustment is permitted. Online 3D-BPP can be naturally formulated as Markov Decision Process (MDP). We adopt deep reinforcement learning, in particular, the on-policy actor-critic framework, to solve this MDP with constrained action space. To learn a practically feasible packing policy, we propose three critical designs. First, we propose an online analysis of packing stability based on a novel stacking tree. It attains a high analysis accuracy while reducing the computational complexity from $O(N^2)$ to $O(N \log N)$, making it especially suited for RL training. Second, we propose a decoupled packing policy learning for different dimensions of placement which enables high-resolution spatial discretization and hence high packing precision. Third, we introduce a reward function that dictates the robot to place items in a far-to-near order and therefore simplifies the collision avoidance in movement planning of the robotic arm. Furthermore, we provide a comprehensive discussion on several key implemental issues. The extensive evaluation demonstrates that our learned policy outperforms the state-of-the-art methods significantly and is practically usable for real-world applications.