Discovering objects and their relations from entangled scene representations

TL;DR

This paper introduces relation networks, a neural architecture capable of learning and reasoning about object relations in structured scenes, facilitating object relation understanding and factorization from complex scene representations.

Contribution

The paper presents relation networks as a versatile neural architecture for object-relation reasoning, capable of learning from scene data and disentangling objects from entangled representations.

Findings

Relation networks can learn object relations from scene descriptions.

They can factorize objects from entangled scene representations.

They enable implicit relation discovery in one-shot learning.

Abstract

Our world can be succinctly and compactly described as structured scenes of objects and relations. A typical room, for example, contains salient objects such as tables, chairs and books, and these objects typically relate to each other by their underlying causes and semantics. This gives rise to correlated features, such as position, function and shape. Humans exploit knowledge of objects and their relations for learning a wide spectrum of tasks, and more generally when learning the structure underlying observed data. In this work, we introduce relation networks (RNs) - a general purpose neural network architecture for object-relation reasoning. We show that RNs are capable of learning object relations from scene description data. Furthermore, we show that RNs can act as a bottleneck that induces the factorization of objects from entangled scene description inputs, and from distributed deep representations of scene images provided by a variational autoencoder. The model can also be used in conjunction with differentiable memory mechanisms for implicit relation discovery in one-shot learning tasks. Our results suggest that relation networks are a potentially powerful architecture for solving a variety of problems that require object relation reasoning.