Weakly Supervised Concept Learning for Object-centric Visual Reasoning

TL;DR

This paper presents a weakly supervised, object-centric approach combining neural perception with symbolic reasoning, achieving high accuracy with minimal labels and strong domain generalization.

Contribution

It introduces a novel weak supervision scheme using a slot-based VAE for object grounding, enabling effective symbolic reasoning with as little as 1% labeled data.

Findings

Reduces supervision to as little as 1% of labels.

Outperforms state-of-the-art foundation models in domain generalization at low supervision.

Successfully discovers complex, abstract rules for object-centric reasoning.

Abstract

Neurosymbolic systems promise to combine deep neural network's (DNN) processing of raw sensor inputs with few-shot performance of symbolic artificial intelligence. Two-stage approaches explicitly decouple DNN based perception from subsequent rule based reasoning. This avoids optimization and interpretability issues of end to end differentiable approaches, but requires costly labels for the perception output. This paper introduces an efficient weak supervision scheme for the perception stage to ground its output symbols for logical induction in object-centric reasoning tasks. It combines a slot-based architecture for object-centricity with a Variational Autoencoder (VAE) for self-supervision, competing with concept guidance on latent dimensions for human interpretable grounding. The resulting predictions are translated into symbolic background knowledge for reasoning frameworks, such as Inductive Logic Programming (ILP), Decision Trees, and Bayesian Networks. Our extensive empirical evaluation on synthetic and real world datasets shows that our approach can discover complex, abstract rules for object centric reasoning whilst reducing supervision to as little as 1% of labels, and being robust even under substantial domain shift. Notably, at 1% supervision it even outperforms state of the art foundation model baselines in domain generalization