Efficient Learning of Harmonic Priors for Pitch Detection in Polyphonic Music

TL;DR

This paper explores the use of physically inspired Gaussian process priors, specifically the Matérn spectral mixture kernel, to improve polyphonic music transcription by better modeling harmonic content and pitch activations.

Contribution

It introduces a novel MSM kernel for frequency modeling and compares sigmoid and softmax regression approaches for pitch detection in polyphonic music.

Findings

MSM kernel effectively models note frequency content.

Learning priors that fit sound event frequencies improves pitch detection.

Cross-correlation between activations is less beneficial than frequency-aligned priors.

Abstract

Automatic music transcription (AMT) aims to infer a latent symbolic representation of a piece of music (piano-roll), given a corresponding observed audio recording. Transcribing polyphonic music (when multiple notes are played simultaneously) is a challenging problem, due to highly structured overlapping between harmonics. We study whether the introduction of physically inspired Gaussian process (GP) priors into audio content analysis models improves the extraction of patterns required for AMT. Audio signals are described as a linear combination of sources. Each source is decomposed into the product of an amplitude-envelope, and a quasi-periodic component process. We introduce the Mat\'ern spectral mixture (MSM) kernel for describing frequency content of singles notes. We consider two different regression approaches. In the sigmoid model every pitch-activation is independently non-linear transformed. In the softmax model several activation GPs are jointly non-linearly transformed. This introduce cross-correlation between activations. We use variational Bayes for approximate inference. We empirically evaluate how these models work in practice transcribing polyphonic music. We demonstrate that rather than encourage dependency between activations, what is relevant for improving pitch detection is to learnt priors that fit the frequency content of the sound events to detect.