Testing GLOM's ability to infer wholes from ambiguous parts

TL;DR

This paper evaluates a simplified GLOM neural network's ability to resolve ambiguity in image parts by forming consistent object representations, demonstrating robustness to noise and transformations.

Contribution

It introduces a simplified GLOM model and shows its effectiveness in resolving ambiguity and maintaining robust object representations.

Findings

Successfully forms consistent object embeddings

Robust to input noise and transformations

Effective in resolving ambiguous parts

Abstract

The GLOM architecture proposed by Hinton [2021] is a recurrent neural network for parsing an image into a hierarchy of wholes and parts. When a part is ambiguous, GLOM assumes that the ambiguity can be resolved by allowing the part to make multi-modal predictions for the pose and identity of the whole to which it belongs and then using attention to similar predictions coming from other possibly ambiguous parts to settle on a common mode that is predicted by several different parts. In this study, we describe a highly simplified version of GLOM that allows us to assess the effectiveness of this way of dealing with ambiguity. Our results show that, with supervised training, GLOM is able to successfully form islands of very similar embedding vectors for all of the locations occupied by the same object and it is also robust to strong noise injections in the input and to out-of-distribution input transformations.