Hybrid Primal Sketch: Combining Analogy, Qualitative Representations, and Computer Vision for Scene Understanding

TL;DR

The paper introduces the Hybrid Primal Sketch, a new framework that integrates computer vision and high-level cognitive models to improve scene understanding and data-efficient learning through analogical reasoning.

Contribution

It presents a novel hybrid framework inspired by Marr's Primal Sketch, combining computer vision and cognitive modeling for enhanced scene analysis.

Findings

Hybrid Primal Sketch effectively produces detailed shape and scene representations.

The framework enables data-efficient learning via analogical generalization.

Preliminary experiments support the framework's potential for diagram understanding.

Abstract

One of the purposes of perception is to bridge between sensors and conceptual understanding. Marr's Primal Sketch combined initial edge-finding with multiple downstream processes to capture aspects of visual perception such as grouping and stereopsis. Given the progress made in multiple areas of AI since then, we have developed a new framework inspired by Marr's work, the Hybrid Primal Sketch, which combines computer vision components into an ensemble to produce sketch-like entities which are then further processed by CogSketch, our model of high-level human vision, to produce both more detailed shape representations and scene representations which can be used for data-efficient learning via analogical generalization. This paper describes our theoretical framework, summarizes several previous experiments, and outlines a new experiment in progress on diagram understanding.