Stabilizing Training with Soft Dynamic Time Warping: A Case Study for Pitch Class Estimation with Weakly Aligned Targets

TL;DR

This paper addresses the instability in training neural networks with Soft Dynamic Time Warping (SDTW) for pitch class estimation, proposing strategies to stabilize soft alignments and improve training robustness.

Contribution

It introduces three novel strategies—hyperparameter scheduling, diagonal prior, and sequence unfolding—to stabilize SDTW-based training with weakly aligned data.

Findings

Strategies improve training stability and alignment accuracy

Experiments demonstrate effectiveness of proposed methods

Discussion on efficiency and implementation challenges

Abstract

Soft dynamic time warping (SDTW) is a differentiable loss function that allows for training neural networks from weakly aligned data. Typically, SDTW is used to iteratively compute and refine soft alignments that compensate for temporal deviations between the training data and its weakly annotated targets. One major problem is that a mismatch between the estimated soft alignments and the reference alignments in the early training stage leads to incorrect parameter updates, making the overall training procedure unstable. In this paper, we investigate such stability issues by considering the task of pitch class estimation from music recordings as an illustrative case study. In particular, we introduce and discuss three conceptually different strategies (a hyperparameter scheduling, a diagonal prior, and a sequence unfolding strategy) with the objective of stabilizing intermediate soft alignment results. Finally, we report on experiments that demonstrate the effectiveness of the strategies and discuss efficiency and implementation issues.