Unraveling Movie Genres through Cross-Attention Fusion of Bi-Modal Synergy of Poster

TL;DR

This paper proposes a novel multi-modal framework that combines visual and textual features from movie posters using cross-attention to improve multilabel movie genre classification, demonstrating superior performance over existing models.

Contribution

The study introduces a cross-attention fusion module for integrating visual and textual poster features, advancing movie genre classification methods.

Findings

Model outperforms several contemporary architectures.

Effective use of OCR-extracted text and visual features.

Promising results on IMDb poster dataset.

Abstract

Movie posters are not just decorative; they are meticulously designed to capture the essence of a movie, such as its genre, storyline, and tone/vibe. For decades, movie posters have graced cinema walls, billboards, and now our digital screens as a form of digital posters. Movie genre classification plays a pivotal role in film marketing, audience engagement, and recommendation systems. Previous explorations into movie genre classification have been mostly examined in plot summaries, subtitles, trailers and movie scenes. Movie posters provide a pre-release tantalizing glimpse into a film's key aspects, which can ignite public interest. In this paper, we presented the framework that exploits movie posters from a visual and textual perspective to address the multilabel movie genre classification problem. Firstly, we extracted text from movie posters using an OCR and retrieved the relevant embedding. Next, we introduce a cross-attention-based fusion module to allocate attention weights to visual and textual embedding. In validating our framework, we utilized 13882 posters sourced from the Internet Movie Database (IMDb). The outcomes of the experiments indicate that our model exhibited promising performance and outperformed even some prominent contemporary architectures.