Physics-Aware Combinatorial Assembly Sequence Planning using Data-free Action Masking

TL;DR

This paper introduces a physics-aware deep reinforcement learning approach with data-free action masking to plan physically valid assembly sequences for combinatorial objects, demonstrated on Lego structures with perfect success rate.

Contribution

It proposes a novel physics-aware online action masking technique for reinforcement learning-based assembly sequence planning, ensuring physical validity without data dependence.

Findings

Achieved 100% success rate on 250+ Lego structures.

Outperformed baseline methods by correctly planning valid sequences.

Ensured physically feasible assembly actions through data-free action masking.

Abstract

Combinatorial assembly uses standardized unit primitives to build objects that satisfy user specifications. This paper studies assembly sequence planning (ASP) for physical combinatorial assembly. Given the shape of the desired object, the goal is to find a sequence of actions for placing unit primitives to build the target object. In particular, we aim to ensure the planned assembly sequence is physically executable. However, ASP for combinatorial assembly is particularly challenging due to its combinatorial nature. To address the challenge, we employ deep reinforcement learning to learn a construction policy for placing unit primitives sequentially to build the desired object. Specifically, we design an online physics-aware action mask that filters out invalid actions, which effectively guides policy learning and ensures violation-free deployment. In the end, we apply the proposed method to Lego assembly with more than 250 3D structures. The experiment results demonstrate that the proposed method plans physically valid assembly sequences to build all structures, achieving a $100\%$ success rate, whereas the best comparable baseline fails more than $40$ structures. Our implementation is available at \url{https://github.com/intelligent-control-lab/PhysicsAwareCombinatorialASP}.