Effective Online 3D Bin Packing with Lookahead Parcels Using Monte Carlo Tree Search

TL;DR

This paper introduces a novel Monte Carlo Tree Search-based framework for online 3D bin packing that leverages lookahead information to adapt to distribution shifts, significantly improving efficiency and robustness in logistics applications.

Contribution

It formulates online 3D bin packing with lookahead parcels as a Model Predictive Control problem and develops a MCTS framework with a dynamic exploration prior and auxiliary reward.

Findings

Achieves over 10% performance gains under distributional shifts

Provides 4% average improvement in online deployment

Demonstrates up to 8% best-case improvement

Abstract

Online 3D Bin Packing (3D-BP) with robotic arms is crucial for reducing transportation and labor costs in modern logistics. While Deep Reinforcement Learning (DRL) has shown strong performance, it often fails to adapt to real-world short-term distribution shifts, which arise as different batches of goods arrive sequentially, causing performance drops. We argue that the short-term lookahead information available in modern logistics systems is key to mitigating this issue, especially during distribution shifts. We formulate online 3D-BP with lookahead parcels as a Model Predictive Control (MPC) problem and adapt the Monte Carlo Tree Search (MCTS) framework to solve it. Our framework employs a dynamic exploration prior that automatically balances a learned RL policy and a robust random policy based on the lookahead characteristics. Additionally, we design an auxiliary reward to penalize long-term spatial waste from individual placements. Extensive experiments on real-world datasets show that our method consistently outperforms state-of-the-art baselines, achieving over 10\% gains under distributional shifts, 4\% average improvement in online deployment, and up to more than 8\% in the best case--demonstrating the effectiveness of our framework.