Online 3D Bin Packing Reinforcement Learning Solution with Buffer

TL;DR

This paper introduces a reinforcement learning framework with a buffer and data augmentation for 3D bin packing, achieving superior space utilization compared to existing methods.

Contribution

It presents a novel RL approach with multi-item action buffer and symmetry-based data augmentation, inspired by AlphaGo, for improved 3D bin packing performance.

Findings

Outperforms state-of-the-art in space utilization

Efficient training with single thread and GPU

Effective use of buffer and data augmentation strategies

Abstract

The 3D Bin Packing Problem (3D-BPP) is one of the most demanded yet challenging problems in industry, where an agent must pack variable size items delivered in sequence into a finite bin with the aim to maximize the space utilization. It represents a strongly NP-Hard optimization problem such that no solution has been offered to date with high performance in space utilization. In this paper, we present a new reinforcement learning (RL) framework for a 3D-BPP solution for improving performance. First, a buffer is introduced to allow multi-item action selection. By increasing the degree of freedom in action selection, a more complex policy that results in better packing performance can be derived. Second, we propose an agnostic data augmentation strategy that exploits both bin item symmetries for improving sample efficiency. Third, we implement a model-based RL method adapted from the popular algorithm AlphaGo, which has shown superhuman performance in zero-sum games. Our adaptation is capable of working in single-player and score based environments. In spite of the fact that AlphaGo versions are known to be computationally heavy, we manage to train the proposed framework with a single thread and GPU, while obtaining a solution that outperforms the state-of-the-art results in space utilization.