Soft-DTW: a Differentiable Loss Function for Time-Series

TL;DR

This paper introduces soft-DTW, a differentiable variant of dynamic time warping, enabling effective learning and clustering of time series by leveraging a smooth, gradient-friendly loss function.

Contribution

The authors develop a differentiable soft-DTW loss function that allows gradient-based optimization for time series tasks, outperforming existing methods in clustering and parameter tuning.

Findings

Soft-DTW is differentiable and computationally efficient.

It improves clustering accuracy under DTW geometry.

It enables effective parameter tuning for time series models.

Abstract

We propose in this paper a differentiable learning loss between time series, building upon the celebrated dynamic time warping (DTW) discrepancy. Unlike the Euclidean distance, DTW can compare time series of variable size and is robust to shifts or dilatations across the time dimension. To compute DTW, one typically solves a minimal-cost alignment problem between two time series using dynamic programming. Our work takes advantage of a smoothed formulation of DTW, called soft-DTW, that computes the soft-minimum of all alignment costs. We show in this paper that soft-DTW is a differentiable loss function, and that both its value and gradient can be computed with quadratic time/space complexity (DTW has quadratic time but linear space complexity). We show that this regularization is particularly well suited to average and cluster time series under the DTW geometry, a task for which our proposal significantly outperforms existing baselines. Next, we propose to tune the parameters of a machine that outputs time series by minimizing its fit with ground-truth labels in a soft-DTW sense.