DDSP-based Neural Waveform Synthesis of Polyphonic Guitar Performance from String-wise MIDI Input

TL;DR

This paper presents a neural waveform synthesis approach for polyphonic guitar from string-wise MIDI input, demonstrating that a simple classification-based control prediction system outperforms more complex architectures.

Contribution

It introduces four neural synthesis systems for guitar, showing the effectiveness of classification over regression for control feature prediction and highlighting the simplest system's superior performance.

Findings

Classification-based control prediction improves synthesis quality.

Simplest system directly predicting synthesis parameters performs best.

Objective and subjective evaluations validate the proposed approach.

Abstract

We explore the use of neural synthesis for acoustic guitar from string-wise MIDI input. We propose four different systems and compare them with both objective metrics and subjective evaluation against natural audio and a sample-based baseline. We iteratively develop these four systems by making various considerations on the architecture and intermediate tasks, such as predicting pitch and loudness control features. We find that formulating the control feature prediction task as a classification task rather than a regression task yields better results. Furthermore, we find that our simplest proposed system, which directly predicts synthesis parameters from MIDI input performs the best out of the four proposed systems. Audio examples are available at https://erl-j.github.io/neural-guitar-web-supplement.