Vision as looking and seeing through a bottleneck

TL;DR

This paper proposes a new perspective on vision as a process constrained by a bottleneck, emphasizing the roles of gaze shifts and neural mechanisms in selecting and recognizing visual information.

Contribution

It introduces a bottleneck framework for understanding vision, highlighting the roles of V1, saliency maps, and gaze shifts in visual recognition.

Findings

V1 initiates the bottleneck and generates a bottom-up saliency map.

Gaze shifts center selected visual content at the fovea.

Top-down feedback refines the recognition process.

Abstract

Progress in vision research has been slower downstream than upstream of primary visual cortex (V1). Traditional frameworks have largely overlooked a central constraint: only a tiny fraction of retinal input is recognized. Thus, to a first approximation, vision is better formulated as looking and seeing through a bottleneck. Looking, mainly by the peripheral visual field, selects visual information to enter this bottleneck, largely via gaze shifts that center selected contents at fovea. Seeing, mainly by the central visual field, recognizes this content. Converging evidence suggests that V1 initiates the bottleneck and contributes to looking by generating a bottom-up saliency map that guides saccades exogenously, and that top-down feedback along the visual pathway, targeting mainly the representation of the central visual field, refines seeing. Progress will accelerate through falsifiable theories that explicitly link behavior with neural substrates, and by experimental designs that avoid forced fixation and precisely track gaze.