ComVi: Context-Aware Optimized Comment Display in Video Playback

TL;DR

ComVi is a system that enhances video viewing by displaying time-synchronized comments based on audio-visual correlation, improving engagement and reducing spoilers.

Contribution

It introduces a novel method to align comments with relevant video moments using audio-visual correlation and optimization for better viewer experience.

Findings

ComVi increased user engagement compared to traditional interfaces.

71.9% of participants preferred ComVi over YouTube and Danmaku.

The system effectively reduces spoilers by contextually displaying comments.

Abstract

On general video-sharing platforms like YouTube, comments are displayed independently of video playback. As viewers often read comments while watching a video, they may encounter ones referring to moments unrelated to the current scene, which can reveal spoilers and disrupt immersion. To address this problem, we present ComVi, a novel system that displays comments at contextually relevant moments, enabling viewers to see time-synchronized comments and video content together. We first map all comments to relevant video timestamps by computing audio-visual correlation, then construct the comment sequence through an optimization that considers temporal relevance, popularity (number of likes), and display duration for comfortable reading. In a user study, ComVi provided a significantly more engaging experience than conventional video interfaces (i.e., YouTube and Danmaku), with 71.9% of participants selecting ComVi as their most preferred interface.