Will You Ever Become Popular? Learning to Predict Virality of Dance Clips

TL;DR

This paper presents a multi-modal deep learning framework for predicting the virality of dance videos, integrating skeletal, appearance, facial, and scenic cues, validated on a large-scale dataset.

Contribution

It introduces novel hierarchical skeleton graph convolution and relational temporal networks for comprehensive dance virality prediction.

Findings

Effective prediction of dance virality demonstrated on VDV dataset

Multi-modal approach outperforms single-modal baselines

Enables applications like personalized recommendations and feedback

Abstract

Dance challenges are going viral in video communities like TikTok nowadays. Once a challenge becomes popular, thousands of short-form videos will be uploaded in merely a couple of days. Therefore, virality prediction from dance challenges is of great commercial value and has a wide range of applications, such as smart recommendation and popularity promotion. In this paper, a novel multi-modal framework which integrates skeletal, holistic appearance, facial and scenic cues is proposed for comprehensive dance virality prediction. To model body movements, we propose a pyramidal skeleton graph convolutional network (PSGCN) which hierarchically refines spatio-temporal skeleton graphs. Meanwhile, we introduce a relational temporal convolutional network (RTCN) to exploit appearance dynamics with non-local temporal relations. An attentive fusion approach is finally proposed to adaptively aggregate predictions from different modalities. To validate our method, we introduce a large-scale viral dance video (VDV) dataset, which contains over 4,000 dance clips of eight viral dance challenges. Extensive experiments on the VDV dataset demonstrate the efficacy of our model. Extensive experiments on the VDV dataset well demonstrate the effectiveness of our approach. Furthermore, we show that short video applications like multi-dimensional recommendation and action feedback can be derived from our model.