Automated Conversion of Music Videos into Lyric Videos

TL;DR

This paper presents an automated pipeline that converts music videos into lyric videos by applying design guidelines to ensure readability and focus, demonstrated through diverse examples and user studies.

Contribution

It introduces a novel automated method for creating lyric videos from music videos, based on carefully designed guidelines for visual harmony and readability.

Findings

Generated lyric videos are effective in maintaining text readability.

The pipeline is robust across diverse input sources.

User studies confirm the quality of the generated videos.

Abstract

Musicians and fans often produce lyric videos, a form of music videos that showcase the song's lyrics, for their favorite songs. However, making such videos can be challenging and time-consuming as the lyrics need to be added in synchrony and visual harmony with the video. Informed by prior work and close examination of existing lyric videos, we propose a set of design guidelines to help creators make such videos. Our guidelines ensure the readability of the lyric text while maintaining a unified focus of attention. We instantiate these guidelines in a fully automated pipeline that converts an input music video into a lyric video. We demonstrate the robustness of our pipeline by generating lyric videos from a diverse range of input sources. A user study shows that lyric videos generated by our pipeline are effective in maintaining text readability and unifying the focus of attention.