Bridging the Gap between Semantics and Multimedia Processing

TL;DR

This paper reviews the semantic gap in multimedia processing and explores combining machine learning with symbolic AI to improve understanding, highlighting challenges and recent advances in the field.

Contribution

It proposes a structured approach integrating machine learning and symbolic AI to bridge the semantic gap in multimedia understanding.

Findings

Combines ML and symbolic AI for multimedia semantics

Highlights challenges in integrating different AI methods

Discusses recent developments in multimedia understanding

Abstract

In this paper, we give an overview of the semantic gap problem in multimedia and discuss how machine learning and symbolic AI can be combined to narrow this gap. We describe the gap in terms of a classical architecture for multimedia processing and discuss a structured approach to bridge it. This approach combines machine learning (for mapping signals to objects) and symbolic AI (for linking objects to meanings). Our main goal is to raise awareness and discuss the challenges involved in this structured approach to multimedia understanding, especially in the view of the latest developments in machine learning and symbolic AI.