Structured Knowledge Representation for Image Retrieval

TL;DR

This paper introduces a formal logic-based framework for semantic image retrieval that combines low-level features with structured descriptions of objects, enabling effective exact and approximate matching.

Contribution

It presents a novel description logic for semantic indexing of images, integrating complex object composition and reasoning services for improved retrieval.

Findings

System achieves comparable results to expert user rankings.

Supports queries by sketch and example.

Demonstrates effective retrieval with similarity measures.

Abstract

We propose a structured approach to the problem of retrieval of images by content and present a description logic that has been devised for the semantic indexing and retrieval of images containing complex objects. As other approaches do, we start from low-level features extracted with image analysis to detect and characterize regions in an image. However, in contrast with feature-based approaches, we provide a syntax to describe segmented regions as basic objects and complex objects as compositions of basic ones. Then we introduce a companion extensional semantics for defining reasoning services, such as retrieval, classification, and subsumption. These services can be used for both exact and approximate matching, using similarity measures. Using our logical approach as a formal specification, we implemented a complete client-server image retrieval system, which allows a user to pose both queries by sketch and queries by example. A set of experiments has been carried out on a testbed of images to assess the retrieval capabilities of the system in comparison with expert users ranking. Results are presented adopting a well-established measure of quality borrowed from textual information retrieval.