TimeNet: Pre-trained deep recurrent neural network for time series classification

TL;DR

TimeNet is a pre-trained deep RNN that learns generalizable features from diverse time series data, significantly improving classification performance across multiple domains without domain-specific training.

Contribution

We introduce TimeNet, a deep RNN trained on multiple domains using seq2seq models, providing a universal feature extractor for time series classification.

Findings

TimeNet embeddings improve classification accuracy.

Pre-trained TimeNet outperforms domain-specific RNNs.

TimeNet surpasses DTW-based nearest neighbor classifiers.

Abstract

Inspired by the tremendous success of deep Convolutional Neural Networks as generic feature extractors for images, we propose TimeNet: a deep recurrent neural network (RNN) trained on diverse time series in an unsupervised manner using sequence to sequence (seq2seq) models to extract features from time series. Rather than relying on data from the problem domain, TimeNet attempts to generalize time series representation across domains by ingesting time series from several domains simultaneously. Once trained, TimeNet can be used as a generic off-the-shelf feature extractor for time series. The representations or embeddings given by a pre-trained TimeNet are found to be useful for time series classification (TSC). For several publicly available datasets from UCR TSC Archive and an industrial telematics sensor data from vehicles, we observe that a classifier learned over the TimeNet embeddings yields significantly better performance compared to (i) a classifier learned over the embeddings given by a domain-specific RNN, as well as (ii) a nearest neighbor classifier based on Dynamic Time Warping.