Application of the SP theory of intelligence to the understanding of natural vision and the development of computer vision

TL;DR

The paper explores how the SP theory of intelligence can be applied to natural and computer vision, emphasizing information compression, recognition, learning, and integration with other sensory modalities.

Contribution

It demonstrates the potential of the SP theory to unify concepts in vision, enabling unsupervised learning, recognition robustness, and integration with broader cognitive functions.

Findings

Edges and corners identified via redundancy extraction

Object recognition using multiple alignment and hierarchy

Robust recognition and learning despite errors

Abstract

The SP theory of intelligence aims to simplify and integrate concepts in computing and cognition, with information compression as a unifying theme. This article discusses how it may be applied to the understanding of natural vision and the development of computer vision. The theory, which is described quite fully elsewhere, is described here in outline but with enough detail to ensure that the rest of the article makes sense. Low level perceptual features such as edges or corners may be identified by the extraction of redundancy in uniform areas in a manner that is comparable with the run-length encoding technique for information compression. The concept of multiple alignment in the SP theory may be applied to the recognition of objects, and to scene analysis, with a hierarchy of parts and sub-parts, and at multiple levels of abstraction. The theory has potential for the unsupervised learning of visual objects and classes of objects, and suggests how coherent concepts may be derived from fragments. As in natural vision, both recognition and learning in the SP system is robust in the face of errors of omission, commission and substitution. The theory suggests how, via vision, we may piece together a knowledge of the three-dimensional structure of objects and of our environment, it provides an account of how we may see things that are not objectively present in an image, and how we recognise something despite variations in the size of its retinal image. And it has things to say about the phenomena of lightness constancy and colour constancy, the role of context in recognition, and ambiguities in visual perception. A strength of the SP theory is that it provides for the integration of vision with other sensory modalities and with other aspects of intelligence.