Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields

TL;DR

This paper develops a theoretical framework for how a generalized Gaussian derivative model of visual receptive fields maintains true covariance under various geometric transformations, enabling invariance in visual perception.

Contribution

It introduces a first-principles derived model that exhibits covariance properties under multiple transformations, linking biological receptive fields to invariance in natural image processing.

Findings

Model obeys covariance under spatial and temporal transformations

Enables handling of multi-view object and event deformations

Implications for biological vision and receptive field variability

Abstract

This paper presents a theory for how geometric image transformations can be handled by a first layer of linear receptive fields, in terms of true covariance properties, which, in turn, enable geometric invariance properties at higher levels in the visual hierarchy. Specifically, we develop this theory for a generalized Gaussian derivative model for visual receptive fields, which is derived in an axiomatic manner from first principles, that reflect symmetry properties of the environment, complemented by structural assumptions to guarantee internally consistent treatment of image structures over multiple spatio-temporal scales. It is shown how the studied generalized Gaussian derivative model for visual receptive fields obeys true covariance properties under spatial scaling transformations, spatial affine transformations, Galilean transformations and temporal scaling transformations, implying that a vision system, based on image and video measurements in terms of the receptive fields according to this model, can to first order of approximation handle the image and video deformations between multiple views of objects delimited by smooth surfaces, as well as between multiple views of spatio-temporal events, under varying relative motions between the objects and events in the world and the observer. We conclude by describing implications of the presented theory for biological vision, regarding connections between the variabilities of the shapes of biological visual receptive fields and the variabilities of spatial and spatio-temporal image structures under natural image transformations.