Disentangling Visual Priors: Unsupervised Learning of Scene Interpretations with Compositional Autoencoder

TL;DR

This paper introduces a neurosymbolic autoencoder that learns to interpret scenes by disentangling visual concepts like objects and transforms using a domain-specific language, enabling better generalization and noise robustness.

Contribution

It presents a novel neurosymbolic architecture combining symbolic priors with neural features to interpret images, which is a new approach in scene understanding.

Findings

Successfully disentangles visual scene aspects

Learns effectively from small datasets

Generalizes well to out-of-sample data

Abstract

Contemporary deep learning architectures lack principled means for capturing and handling fundamental visual concepts, like objects, shapes, geometric transforms, and other higher-level structures. We propose a neurosymbolic architecture that uses a domain-specific language to capture selected priors of image formation, including object shape, appearance, categorization, and geometric transforms. We express template programs in that language and learn their parameterization with features extracted from the scene by a convolutional neural network. When executed, the parameterized program produces geometric primitives which are rendered and assessed for correspondence with the scene content and trained via auto-association with gradient. We confront our approach with a baseline method on a synthetic benchmark and demonstrate its capacity to disentangle selected aspects of the image formation process, learn from small data, correct inference in the presence of noise, and out-of-sample generalization.