Learning Discriminative Prototypes with Dynamic Time Warping

TL;DR

This paper introduces DP-DTW, a novel method that learns class-specific prototypes for temporal data, improving classification and enabling detailed reasoning and summarization in video analysis.

Contribution

The paper proposes DP-DTW, which learns discriminative prototypes for temporal recognition and integrates with deep learning for weakly supervised action segmentation.

Findings

DP-DTW outperforms conventional DTW on time series benchmarks.

DP-DTW achieves state-of-the-art results in weakly supervised action segmentation.

Learned prototypes enable detailed reasoning and video summarization.

Abstract

Dynamic Time Warping (DTW) is widely used for temporal data processing. However, existing methods can neither learn the discriminative prototypes of different classes nor exploit such prototypes for further analysis. We propose Discriminative Prototype DTW (DP-DTW), a novel method to learn class-specific discriminative prototypes for temporal recognition tasks. DP-DTW shows superior performance compared to conventional DTWs on time series classification benchmarks. Combined with end-to-end deep learning, DP-DTW can handle challenging weakly supervised action segmentation problems and achieves state of the art results on standard benchmarks. Moreover, detailed reasoning on the input video is enabled by the learned action prototypes. Specifically, an action-based video summarization can be obtained by aligning the input sequence with action prototypes.