Translating Paintings Into Music Using Neural Networks

TL;DR

This paper introduces a neural network-based system that translates paintings into music, enabling real-time artistic collaborations and inspiring improvisation in live performances.

Contribution

It presents a novel neural network approach for translating visual art into music, facilitating real-time artistic interactions and performances.

Findings

System successfully generates music from paintings

Enables real-time artistic translation and collaboration

Provides a new tool for live performance and improvisation

Abstract

We propose a system that learns from artistic pairings of music and corresponding album cover art. The goal is to 'translate' paintings into music and, in further stages of development, the converse. We aim to deploy this system as an artistic tool for real time 'translations' between musicians and painters. The system's outputs serve as elements to be employed in a joint live performance of music and painting, or as generative material to be used by the artists as inspiration for their improvisation.