Personalizing Fast-Forward Videos Based on Visual and Textual Features from Social Network

TL;DR

This paper introduces a personalized fast-forward video method for First-Person Videos that leverages social network text data to tailor content to user interests, significantly improving relevance and viewer engagement.

Contribution

The work presents a novel approach combining visual and textual social network data to personalize fast-forwarding in FPVs, outperforming existing methods in accuracy.

Findings

Achieved up to 12.8% higher F1 score than competitors.

Demonstrated effectiveness through extensive experiments and user study.

Successfully integrated social media text data for personalized video summarization.

Abstract

The growth of Social Networks has fueled the habit of people logging their day-to-day activities, and long First-Person Videos (FPVs) are one of the main tools in this new habit. Semantic-aware fast-forward methods are able to decrease the watch time and select meaningful moments, which is key to increase the chances of these videos being watched. However, these methods can not handle semantics in terms of personalization. In this work, we present a new approach to automatically creating personalized fast-forward videos for FPVs. Our approach explores the availability of text-centric data from the user's social networks such as status updates to infer her/his topics of interest and assigns scores to the input frames according to her/his preferences. Extensive experiments are conducted on three different datasets with simulated and real-world users as input, achieving an average F1 score of up to 12.8 percentage points higher than the best competitors. We also present a user study to demonstrate the effectiveness of our method.