My House, My Rules: Learning Tidying Preferences with Graph Neural Networks

TL;DR

This paper introduces NeatNet, a graph neural network-based variational autoencoder that learns individual user preferences for arranging household objects, enabling personalized and neat arrangements in robotic tidying tasks.

Contribution

The paper presents NeatNet, a novel GNN-based VAE architecture that models subjective user preferences for object arrangements, with generalization to new objects using word embeddings.

Findings

Consistently produces personalized arrangements

Successfully generalizes to new objects

Demonstrates effectiveness across various scenarios

Abstract

Robots that arrange household objects should do so according to the user's preferences, which are inherently subjective and difficult to model. We present NeatNet: a novel Variational Autoencoder architecture using Graph Neural Network layers, which can extract a low-dimensional latent preference vector from a user by observing how they arrange scenes. Given any set of objects, this vector can then be used to generate an arrangement which is tailored to that user's spatial preferences, with word embeddings used for generalisation to new objects. We develop a tidying simulator to gather rearrangement examples from 75 users, and demonstrate empirically that our method consistently produces neat and personalised arrangements across a variety of rearrangement scenarios.