Polyphonic Pitch Tracking with Deep Layered Learning

TL;DR

This paper introduces a deep layered neural network system for polyphonic pitch tracking that accurately estimates fundamental frequencies, onsets, and offsets, achieving state-of-the-art results across multiple datasets.

Contribution

The paper presents a novel deep layered learning framework combining cascading neural networks and iterative note correction for improved polyphonic pitch tracking.

Findings

Achieved state-of-the-art results on four public datasets.

Effectively tracks f0, onsets, and offsets in polyphonic audio.

System performs well across diverse musical instruments.

Abstract

This paper presents a polyphonic pitch tracking system able to extract both framewise and note-based estimates from audio. The system uses several artificial neural networks in a deep layered learning setup. First, cascading networks are applied to a spectrogram for framewise fundamental frequency (f0) estimation. A sparse receptive field is learned by the first network and then used as a filter kernel for parameter sharing throughout the system. The f0 activations are connected across time to extract pitch contours. These contours define a framework within which subsequent networks perform onset and offset detection, operating across both time and smaller pitch fluctuations at the same time. As input, the networks use, e.g., variations of latent representations from the f0 estimation network. Finally, incorrect tentative notes are removed one by one in an iterative procedure that allows a network to classify notes within an accurate context. The system was evaluated on four public test sets: MAPS, Bach10, TRIOS, and the MIREX Woodwind quintet, and performed state-of-the-art results for all four datasets. It performs well across all subtasks: f0, pitched onset, and pitched offset tracking.