Dynamic Time Warp Convolutional Networks

TL;DR

This paper introduces a novel convolutional layer that incorporates Dynamic Time Warping principles to better handle local temporal deformations in sequence data, improving classification accuracy.

Contribution

It proposes a non-parametric warping convolutional layer inspired by DTW, enhancing deep networks' ability to model aligned temporal sequences.

Findings

Outperforms standard 1-D convolution in accuracy on time series tasks

Supports integration with various neural network architectures

Hyperparameter analysis confirms robustness across datasets

Abstract

Where dealing with temporal sequences it is fair to assume that the same kind of deformations that motivated the development of the Dynamic Time Warp algorithm could be relevant also in the calculation of the dot product ("convolution") in a 1-D convolution layer. In this work a method is proposed for aligning the convolution filter and the input where they are locally out of phase utilising an algorithm similar to the Dynamic Time Warp. The proposed method enables embedding a non-parametric warping of temporal sequences for increasing similarity directly in deep networks and can expand on the generalisation capabilities and the capacity of standard 1-D convolution layer where local sequential deformations are present in the input. Experimental results demonstrate the proposed method exceeds or matches the standard 1-D convolution layer in terms of the maximum accuracy achieved on a number of time series classification tasks. In addition the impact of different hyperparameters settings is investigated given different datasets and the results support the conclusions of previous work done in relation to the choice of DTW parameter values. The proposed layer can be freely integrated with other typical layers to compose deep artificial neural networks of an arbitrary architecture that are trained using standard stochastic gradient descent.