ViolinDiff: Enhancing Expressive Violin Synthesis with Pitch Bend Conditioning

TL;DR

ViolinDiff introduces a diffusion-based framework that explicitly models pitch bend contours from MIDI data to produce more realistic and expressive violin sounds, addressing the challenge of polyphonic F0 contour synthesis.

Contribution

The paper presents a novel two-stage diffusion model that explicitly incorporates pitch bend information for expressive violin sound synthesis from MIDI files.

Findings

Generated violin sounds are more realistic with pitch bend modeling.

Quantitative metrics show improved synthesis quality.

Listening tests favor the proposed method over baselines.

Abstract

Modeling the natural contour of fundamental frequency (F0) plays a critical role in music audio synthesis. However, transcribing and managing multiple F0 contours in polyphonic music is challenging, and explicit F0 contour modeling has not yet been explored for polyphonic instrumental synthesis. In this paper, we present ViolinDiff, a two-stage diffusion-based synthesis framework. For a given violin MIDI file, the first stage estimates the F0 contour as pitch bend information, and the second stage generates mel spectrogram incorporating these expressive details. The quantitative metrics and listening test results show that the proposed model generates more realistic violin sounds than the model without explicit pitch bend modeling. Audio samples are available online: daewoung.github.io/ViolinDiff-Demo.