Multimodal Content Representation and Similarity Ranking of Movies

TL;DR

This paper explores multi-modal content features of movies, including subtitles, audio, and metadata, to develop models for movie similarity ranking and recommendation, introducing a novel topic model browser and retrieval system.

Contribution

It presents a new multi-modal representation approach for movies, combining subtitles, audio, and metadata, and introduces a novel topic model browser for exploring movie similarities.

Findings

Successful extraction of multi-modal features from movie content

Effective movie similarity ranking using fusion models

Development of a novel movie topic model browser

Abstract

In this paper we examine the existence of correlation between movie similarity and low level features from respective movie content. In particular, we demonstrate the extraction of multi-modal representation models of movies based on subtitles, audio and metadata mining. We emphasize our research in topic modeling of movies based on their subtitles. In order to demonstrate the proposed content representation approach, we have built a small dataset of 160 widely known movies. We assert movie similarities, as propagated by the singular modalities and fusion models, in the form of recommendation rankings. We showcase a novel topic model browser for movies that allows for exploration of the different aspects of similarities between movies and an information retrieval system for movie similarity based on multi-modal content.