Master Rules from Chaos: Learning to Reason, Plan, and Interact from Chaos for Tangram Assembly

TL;DR

This paper introduces MRChaos, a novel learning-based approach enabling robots to assemble tangram objects and similar items by self-exploration in simulation, without prior models, demonstrating strong generalization capabilities.

Contribution

MRChaos is a new method that learns to assemble objects from chaos using visual feedback, surpassing traditional geometric models and enabling generalization to unseen objects.

Findings

MRChaos successfully assembles novel tangram objects in simulation.

The approach generalizes to unseen objects with minimal prompts.

It demonstrates potential in wider applications like cutlery assembly.

Abstract

Tangram assembly, the art of human intelligence and manipulation dexterity, is a new challenge for robotics and reveals the limitations of state-of-the-arts. Here, we describe our initial exploration and highlight key problems in reasoning, planning, and manipulation for robotic tangram assembly. We present MRChaos (Master Rules from Chaos), a robust and general solution for learning assembly policies that can generalize to novel objects. In contrast to conventional methods based on prior geometric and kinematic models, MRChaos learns to assemble randomly generated objects through self-exploration in simulation without prior experience in assembling target objects. The reward signal is obtained from the visual observation change without manually designed models or annotations. MRChaos retains its robustness in assembling various novel tangram objects that have never been encountered during training, with only silhouette prompts. We show the potential of MRChaos in wider applications such as cutlery combinations. The presented work indicates that radical generalization in robotic assembly can be achieved by learning in much simpler domains.