Emergence of foveal image sampling from learning to attend in visual scenes

TL;DR

This paper introduces a neural attention model with a learnable retinal sampling lattice that, after training on a visual search task, naturally develops a fovea-like high-resolution center and low-resolution periphery, mirroring primate retinal structure.

Contribution

It demonstrates how a neural attention model can spontaneously develop a foveal sampling pattern similar to biological retinas through learning.

Findings

The model's retinal lattice resembles the primate retina's eccentricity-dependent sampling.

Emergent properties can be amplified or suppressed by changing training conditions.

The model effectively performs visual search with minimal fixations.

Abstract

We describe a neural attention model with a learnable retinal sampling lattice. The model is trained on a visual search task requiring the classification of an object embedded in a visual scene amidst background distractors using the smallest number of fixations. We explore the tiling properties that emerge in the model's retinal sampling lattice after training. Specifically, we show that this lattice resembles the eccentricity dependent sampling lattice of the primate retina, with a high resolution region in the fovea surrounded by a low resolution periphery. Furthermore, we find conditions where these emergent properties are amplified or eliminated providing clues to their function.