ITO-Master: Inference-Time Optimization for Audio Effects Modeling of Music Mastering Processors

TL;DR

The paper introduces ITO-Master, a novel inference-time optimization framework for audio mastering style transfer that enhances user control and improves the fidelity of the transferred mastering style.

Contribution

It presents a new inference-time optimization method for mastering style transfer, enabling dynamic user adjustments and improved style similarity in audio processing.

Findings

ITO improves style transfer accuracy

User control enhances mastering results

Both black-box and white-box models benefit from ITO

Abstract

Music mastering style transfer aims to model and apply the mastering characteristics of a reference track to a target track, simulating the professional mastering process. However, existing methods apply fixed processing based on a reference track, limiting users' ability to fine-tune the results to match their artistic intent. In this paper, we introduce the ITO-Master framework, a reference-based mastering style transfer system that integrates Inference-Time Optimization (ITO) to enable finer user control over the mastering process. By optimizing the reference embedding during inference, our approach allows users to refine the output dynamically, making micro-level adjustments to achieve more precise mastering results. We explore both black-box and white-box methods for modeling mastering processors and demonstrate that ITO improves mastering performance across different styles. Through objective evaluation, subjective listening tests, and qualitative analysis using text-based conditioning with CLAP embeddings, we validate that ITO enhances mastering style similarity while offering increased adaptability. Our framework provides an effective and user-controllable solution for mastering style transfer, allowing users to refine their results beyond the initial style transfer.