A Topic Model Approach to Multi-Modal Similarity

TL;DR

This paper introduces a multi-modal topic model to measure similarity between heterogeneous objects, evaluated on music data, and compares different model realizations using the Mantel test.

Contribution

It presents a novel approach for multi-modal similarity measurement using topic models and introduces a method to compare model realizations with the Mantel test.

Findings

Effective similarity measurement on music data

Comparison of model realizations using Mantel test

Potential for improved multimedia object retrieval

Abstract

Calculating similarities between objects defined by many heterogeneous data modalities is an important challenge in many multimedia applications. We use a multi-modal topic model as a basis for defining such a similarity between objects. We propose to compare the resulting similarities from different model realizations using the non-parametric Mantel test. The approach is evaluated on a music dataset.