Solving Visual Analogies Using Neural Algorithmic Reasoning

TL;DR

This paper introduces a neural reasoning approach to solve visual analogies by learning sequences of neural transformations, enabling generalization to unseen shapes and positions.

Contribution

It proposes a neural algorithmic reasoning method for visual analogies, extending symbolic search with neural transformations for better generalization.

Findings

The neural approach generalizes to unseen shapes and positions.

It outperforms symbolic search in analogy tasks.

Neural transformations effectively manipulate image representations.

Abstract

We consider a class of visual analogical reasoning problems that involve discovering the sequence of transformations by which pairs of input/output images are related, so as to analogously transform future inputs. This program synthesis task can be easily solved via symbolic search. Using a variation of the `neural analogical reasoning' approach of (Velickovic and Blundell 2021), we instead search for a sequence of elementary neural network transformations that manipulate distributed representations derived from a symbolic space, to which input images are directly encoded. We evaluate the extent to which our `neural reasoning' approach generalizes for images with unseen shapes and positions.