Glow with the Flow: AI-Assisted Creation of Ambient Lightscapes for Music Videos

TL;DR

This paper introduces an AI-assisted system that generates ambient light sequences for music videos, making designed lighting more accessible for non-professional creators by providing a viable initial design based on extracted audio-visual features.

Contribution

The work presents a novel AI system that automates the creation of ambient light effects for music videos, informed by professional design heuristics and evaluated against human-created designs.

Findings

AI-generated designs serve as effective starting points for human refinement.

Participants found the system's output to be a viable baseline for further editing.

The approach supports broader adoption of designed lighting in music visualization.

Abstract

Designed light is an established modality for live performance and music playback. Despite the growing availability of consumer smart lighting, the creation of designed light for music visualization remains limited to professional contexts due to time and skill constraints. To address this, we present an AI-assisted system for generating ambient light sequences for music videos. Informed by professional design heuristics, the system extracts salient features from source video and audio to generate an editable preliminary design of object based ambient light effect. We evaluated the system by comparing its autonomous output against hand-authored designs for three music videos. Findings from responses by 32 participants indicate that the initial output provides a viable baseline for further refinement by human authors. This work demonstrates the utility of AI-assisted workflows in supporting the creation and adoption of designed light beyond professional venues.