Contrastive Training of Complex-Valued Autoencoders for Object Discovery

TL;DR

This paper introduces a novel contrastive training method for complex-valued autoencoders that enhances object discovery capabilities, allowing unsupervised detection of multiple objects in color datasets beyond previous limitations.

Contribution

We develop architectural modifications and a contrastive learning approach that enable synchrony-based models to discover multiple objects in color datasets without supervision.

Findings

Successfully discover multiple objects in color datasets

Operate in an unsupervised manner

Handle more than three objects simultaneously

Abstract

Current state-of-the-art object-centric models use slots and attention-based routing for binding. However, this class of models has several conceptual limitations: the number of slots is hardwired; all slots have equal capacity; training has high computational cost; there are no object-level relational factors within slots. Synchrony-based models in principle can address these limitations by using complex-valued activations which store binding information in their phase components. However, working examples of such synchrony-based models have been developed only very recently, and are still limited to toy grayscale datasets and simultaneous storage of less than three objects in practice. Here we introduce architectural modifications and a novel contrastive learning method that greatly improve the state-of-the-art synchrony-based model. For the first time, we obtain a class of synchrony-based models capable of discovering objects in an unsupervised manner in multi-object color datasets and simultaneously representing more than three objects.