METEOR: Melody-aware Texture-controllable Symbolic Orchestral Music Generation via Transformer VAE

TL;DR

METEOR is a Transformer VAE model that enables melody-aware, texture-controllable re-orchestration of symbolic music, preserving melodic fidelity while allowing detailed texture and instrumentation control.

Contribution

It introduces a novel Transformer-based VAE for symbolic re-orchestration with controllable texture and melody preservation, outperforming existing style transfer models.

Findings

Outperforms style transfer models in quality and controllability

Achieves zero-shot orchestration comparable to specialized models

Maintains melodic fidelity with texture control

Abstract

Re-orchestration is the process of adapting a music piece for a different set of instruments. By altering the original instrumentation, the orchestrator often modifies the musical texture while preserving a recognizable melodic line and ensures that each part is playable within the technical and expressive capabilities of the chosen instruments. In this work, we propose METEOR, a model for generating Melody-aware Texture-controllable re-Orchestration with a Transformer-based variational auto-encoder (VAE). This model performs symbolic instrumental and textural music style transfers with a focus on melodic fidelity and controllability. We allow bar- and track-level controllability of the accompaniment with various textural attributes while keeping a homophonic texture. With both subjective and objective evaluations, we show that our model outperforms style transfer models on a re-orchestration task in terms of generation quality and controllability. Moreover, it can be adapted for a lead sheet orchestration task as a zero-shot learning model, achieving performance comparable to a model specifically trained for this task.