A Multimodal Approach to Predict Social Media Popularity

TL;DR

This paper introduces a new multimodal dataset and approach for predicting social media photo popularity by combining visual, textual, and social features, achieving competitive results with less training data.

Contribution

It presents a novel multimodal dataset for social media popularity prediction and a combined feature approach that performs well with reduced training data.

Findings

Multimodal features improve prediction accuracy.

The approach achieves comparable results with less data.

Combining visual, textual, and social features is effective.

Abstract

Multiple modalities represent different aspects by which information is conveyed by a data source. Modern day social media platforms are one of the primary sources of multimodal data, where users use different modes of expression by posting textual as well as multimedia content such as images and videos for sharing information. Multimodal information embedded in such posts could be useful in predicting their popularity. To the best of our knowledge, no such multimodal dataset exists for the prediction of social media photos. In this work, we propose a multimodal dataset consisiting of content, context, and social information for popularity prediction. Specifically, we augment the SMPT1 dataset for social media prediction in ACM Multimedia grand challenge 2017 with image content, titles, descriptions, and tags. Next, in this paper, we propose a multimodal approach which exploits visual features (i.e., content information), textual features (i.e., contextual information), and social features (e.g., average views and group counts) to predict popularity of social media photos in terms of view counts. Experimental results confirm that despite our multimodal approach uses the half of the training dataset from SMP-T1, it achieves comparable performance with that of state-of-the-art.