# Learning to Assemble the Soma Cube with Legal-Action Masked DQN and Safe ZYZ Regrasp on a Doosan M0609

## TL;DR

This paper introduces a novel constraint-aware reinforcement learning approach combining legal-action masking and safe regrasp strategies for autonomous Soma cube assembly on a collaborative robot, achieving high success rates efficiently.

## Contribution

It is the first to integrate legal-action masked DQN with safe ZYZ regrasp and hierarchical decomposition for robotic assembly learning on a collaborative robot.

## Key findings

- Achieved 100% success on simple assembly within 500 episodes.
- Reduced computational complexity significantly with hierarchical Q-function.
- Demonstrated effective curriculum learning across multiple difficulty levels.

## Abstract

This paper presents the first comprehensive application of legal-action masked Deep Q-Networks with safe ZYZ regrasp strategies to an underactuated gripper-equipped 6-DOF collaborative robot for autonomous Soma cube assembly learning. Our approach represents the first systematic integration of constraint-aware reinforcement learning with singularity-safe motion planning on a Doosan M0609 collaborative robot. We address critical challenges in robotic manipulation: combinatorial action space explosion, unsafe motion planning, and systematic assembly strategy learning. Our system integrates a legal-action masked DQN with hierarchical architecture that decomposes Q-function estimation into orientation and position components, reducing computational complexity from $O(3,132)$ to $O(116) + O(27)$ while maintaining solution completeness. The robot-friendly reward function encourages ground-first, vertically accessible assembly sequences aligned with manipulation constraints. Curriculum learning across three progressive difficulty levels (2-piece, 3-piece, 7-piece) achieves remarkable training efficiency: 100\% success rate for Level 1 within 500 episodes, 92.9\% for Level 2, and 39.9\% for Level 3 over 105,300 total training episodes.