Semantic Image Retrieval by Uniting Deep Neural Networks and Cognitive Architectures

TL;DR

This paper presents a hybrid approach combining deep neural networks and cognitive architectures to improve semantic image and video retrieval, enabling complex query execution involving object detection and spatial arrangements.

Contribution

It introduces a novel hybrid system integrating YOLOv2 and OpenCog for semantic retrieval, bridging the gap between deep learning and cognitive reasoning.

Findings

Effective retrieval of video frames based on object classes and spatial arrangements

Successful integration of deep neural networks with cognitive architectures

Enhanced capabilities for complex semantic queries

Abstract

Image and video retrieval by their semantic content has been an important and challenging task for years, because it ultimately requires bridging the symbolic/subsymbolic gap. Recent successes in deep learning enabled detection of objects belonging to many classes greatly outperforming traditional computer vision techniques. However, deep learning solutions capable of executing retrieval queries are still not available. We propose a hybrid solution consisting of a deep neural network for object detection and a cognitive architecture for query execution. Specifically, we use YOLOv2 and OpenCog. Queries allowing the retrieval of video frames containing objects of specified classes and specified spatial arrangement are implemented.