An Audio-Driven System For Real-Time Music Visualisation
Max Graf, Harold Chijioke Opara, Mathieu Barthet

TL;DR
This paper introduces a modular system for real-time music visualization that uses audio features and mapping interfaces to enhance audiovisual experiences, with positive user feedback on its effectiveness and usability.
Contribution
The paper presents a novel modular architecture for real-time audio-driven visualizations, integrating audio feature extraction, a 3D environment, and a visual mapping interface.
Findings
Visualizations improved music complementarity over baseline
System received above-average Creativity Support Index
Usability scores indicate room for interface improvements
Abstract
Computer-generated visualisations can accompany recorded or live music to create novel audiovisual experiences for audiences. We present a system to streamline the creation of audio-driven visualisations based on audio feature extraction and mapping interfaces. Its architecture is based on three modular software components: backend (audio plugin), frontend (3D game-like environment), and middleware (visual mapping interface). We conducted a user evaluation comprising two stages. Results from the first stage (34 participants) indicate that music visualisations generated with the system were significantly better at complementing the music than a baseline visualisation. Nine participants took part in the second stage involving interactive tasks. Overall, the system yielded a Creativity Support Index above average (68.1) and a System Usability Scale index (58.6) suggesting that ease of use…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMusic and Audio Processing · Music Technology and Sound Studies · Speech and Audio Processing
