# Automatic Realistic Music Video Generation from Segments of Youtube Videos

## TL;DR

This paper introduces an automated method to generate realistic music videos from YouTube segments by matching music genre and visual color themes, producing videos often mistaken for professional work.

## Contribution

The novel approach automatically creates high-quality music videos by combining genre analysis, color clustering, and synchronization with musical boundaries, mimicking professional MVs.

## Key findings

- 45% of generated videos are mistaken for professional MVs
- 21.6% are mistaken for amateur-made MVs
- The method effectively aligns visual segments with musical changes

## Abstract

A Music Video (MV) is a video aiming at visually illustrating or extending the meaning of its background music. This paper proposes a novel method to automatically generate, from an input music track, a music video made of segments of Youtube music videos which would fit this music. The system analyzes the input music to find its genre (pop, rock, ...) and finds segmented MVs with the same genre in the database. Then, a K-Means clustering is done to group video segments by color histogram, meaning segments of MVs having the same global distribution of colors. A few clusters are randomly selected, then are assembled around music boundaries, which are moments where a significant change in the music occurs (for instance, transitioning from verse to chorus). This way, when the music changes, the video color mood changes as well. This work aims at generating high-quality realistic MVs, which could be mistaken for man-made MVs. By asking users to identify, in a batch of music videos containing professional MVs, amateur-made MVs and generated MVs by our algorithm, we show that our algorithm gives satisfying results, as 45% of generated videos are mistaken for professional MVs and 21.6% are mistaken for amateur-made MVs. More information can be found in the project website: http://ml.cs.tsinghua.edu.cn/~sarah/