MusicAIR: A Multimodal AI Music Generation Framework Powered by an Algorithm-Driven Core

TL;DR

MusicAIR introduces a novel multimodal AI framework that generates coherent, human-like music from lyrics, text, or images using an algorithm-driven core, reducing copyright risks and enhancing accessibility for musicians.

Contribution

The paper presents a new multimodal AI music generation framework with an algorithm-driven symbolic core, enabling copyright-safe, theory-compliant music creation from multiple input modalities.

Findings

Achieves an average key confidence of 85%, surpassing human composers at 79%.

Generates diverse, human-like music compositions.

Supports lyric-to-song, text-to-music, and image-to-music generation.

Abstract

Recent advances in generative AI have made music generation a prominent research focus. However, many neural-based models rely on large datasets, raising concerns about copyright infringement and high-performance costs. In contrast, we propose MusicAIR, an innovative multimodal AI music generation framework powered by a novel algorithm-driven symbolic music core, effectively mitigating copyright infringement risks. The music core algorithms connect critical lyrical and rhythmic information to automatically derive musical features, creating a complete, coherent melodic score solely from the lyrics. The MusicAIR framework facilitates music generation from lyrics, text, and images. The generated score adheres to established principles of music theory, lyrical structure, and rhythmic conventions. We developed Generate AI Music (GenAIM), a web tool using MusicAIR for lyric-to-song, text-to-music, and image-to-music generation. In our experiments, we evaluated AI-generated music scores produced by the system using both standard music metrics and innovative analysis that compares these compositions with original works. The system achieves an average key confidence of 85%, outperforming human composers at 79%, and aligns closely with established music theory standards, demonstrating its ability to generate diverse, human-like compositions. As a co-pilot tool, GenAIM can serve as a reliable music composition assistant and a possible educational composition tutor while simultaneously lowering the entry barrier for all aspiring musicians, which is innovative and significantly contributes to AI for music generation.