Device-Guided Music Transfer
Manh Pham Hung, Changshuo Hu, Ting Dang, Dong Ma

TL;DR
DeMT is a novel method that uses a vision-language model to extract device-specific embeddings from speaker frequency responses, enabling effective device-style transfer and adaptation in music playback.
Contribution
It introduces a new approach combining vision-language models and transformers to adapt music to unseen speaker devices based on their frequency response curves.
Findings
Enables effective speaker-style transfer.
Supports robust few-shot adaptation for unseen devices.
Improves quality and style augmentation in music playback.
Abstract
Device-guided music transfer adapts playback across unseen devices for users who lack them. Existing methods mainly focus on modifying the timbre, rhythm, harmony, or instrumentation to mimic genres or artists, overlooking the diverse hardware properties of the playback device (i.e., speaker). Therefore, we propose DeMT, which processes a speaker's frequency response curve as a line graph using a vision-language model to extract device embeddings. These embeddings then condition a hybrid transformer via feature-wise linear modulation. Fine-tuned on a self-collected dataset, DeMT enables effective speaker-style transfer and robust few-shot adaptation for unseen devices, supporting applications like device-style augmentation and quality enhancement.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpeech and Audio Processing · Music and Audio Processing · Music Technology and Sound Studies
