Techniques for Symbol Grounding with SATNet

TL;DR

This paper introduces a self-supervised pre-training pipeline and proofreading method for SATNet, enabling it to solve visual reasoning problems without explicit supervision and overcoming the symbol grounding challenge.

Contribution

It presents a novel self-supervised pre-training approach and proofreading technique that enhance SATNet's ability to perform visual reasoning without label leakage.

Findings

SATNet achieves full accuracy on challenging datasets without label supervision.

The proposed methods outperform previous state-of-the-art on Visual Sudoku.

Self-supervised training broadens SATNet's applicability to unlabeled visual reasoning tasks.

Abstract

Many experts argue that the future of artificial intelligence is limited by the field's ability to integrate symbolic logical reasoning into deep learning architectures. The recently proposed differentiable MAXSAT solver, SATNet, was a breakthrough in its capacity to integrate with a traditional neural network and solve visual reasoning problems. For instance, it can learn the rules of Sudoku purely from image examples. Despite its success, SATNet was shown to succumb to a key challenge in neurosymbolic systems known as the Symbol Grounding Problem: the inability to map visual inputs to symbolic variables without explicit supervision ("label leakage"). In this work, we present a self-supervised pre-training pipeline that enables SATNet to overcome this limitation, thus broadening the class of problems that SATNet architectures can solve to include datasets where no intermediary labels are available at all. We demonstrate that our method allows SATNet to attain full accuracy even with a harder problem setup that prevents any label leakage. We additionally introduce a proofreading method that further improves the performance of SATNet architectures, beating the state-of-the-art on Visual Sudoku.