Neural Systematic Binder

TL;DR

The paper introduces SysBinder, a neural mechanism that constructs structured, object-centric scene representations from unstructured images, enabling systematic generalization and improved factor disentanglement.

Contribution

It proposes a novel Block-Slot Representation and a neural layer, SysBinder, for unsupervised, systematic, and general-purpose scene understanding across modalities.

Findings

SysBinder outperforms conventional methods in factor disentanglement.

It enables systematic generalization to unseen factor combinations.

Effective on complex scene images like CLEVR-Tex.

Abstract

The key to high-level cognition is believed to be the ability to systematically manipulate and compose knowledge pieces. While token-like structured knowledge representations are naturally provided in text, it is elusive how to obtain them for unstructured modalities such as scene images. In this paper, we propose a neural mechanism called Neural Systematic Binder or SysBinder for constructing a novel structured representation called Block-Slot Representation. In Block-Slot Representation, object-centric representations known as slots are constructed by composing a set of independent factor representations called blocks, to facilitate systematic generalization. SysBinder obtains this structure in an unsupervised way by alternatingly applying two different binding principles: spatial binding for spatial modularity across the full scene and factor binding for factor modularity within an object. SysBinder is a simple, deterministic, and general-purpose layer that can be applied as a drop-in module in any arbitrary neural network and on any modality. In experiments, we find that SysBinder provides significantly better factor disentanglement within the slots than the conventional object-centric methods, including, for the first time, in visually complex scene images such as CLEVR-Tex. Furthermore, we demonstrate factor-level systematicity in controlled scene generation by decoding unseen factor combinations.