LEGO-Net: Learning Regular Rearrangements of Objects in Rooms

TL;DR

LEGO-Net is a transformer-based iterative model that learns to automatically rearrange objects in messy rooms into regular, aesthetically pleasing arrangements by reducing disorder through a diffusion-inspired process.

Contribution

The paper introduces LEGO-Net, a novel data-driven method for rearranging existing messy scenes into regular arrangements without needing explicit goal states, inspired by diffusion models.

Findings

LEGO-Net outperforms existing methods in rearranging room scenes.

The model reliably recovers regular arrangements from perturbed scenes.

A new metric for evaluating arrangement regularity is proposed.

Abstract

Humans universally dislike the task of cleaning up a messy room. If machines were to help us with this task, they must understand human criteria for regular arrangements, such as several types of symmetry, co-linearity or co-circularity, spacing uniformity in linear or circular patterns, and further inter-object relationships that relate to style and functionality. Previous approaches for this task relied on human input to explicitly specify goal state, or synthesized scenes from scratch -- but such methods do not address the rearrangement of existing messy scenes without providing a goal state. In this paper, we present LEGO-Net, a data-driven transformer-based iterative method for LEarning reGular rearrangement of Objects in messy rooms. LEGO-Net is partly inspired by diffusion models -- it starts with an initial messy state and iteratively ''de-noises'' the position and orientation of objects to a regular state while reducing distance traveled. Given randomly perturbed object positions and orientations in an existing dataset of professionally-arranged scenes, our method is trained to recover a regular re-arrangement. Results demonstrate that our method is able to reliably rearrange room scenes and outperform other methods. We additionally propose a metric for evaluating regularity in room arrangements using number-theoretic machinery.