Towards a universal neural network encoder for time series

TL;DR

This paper introduces a convolutional neural network-based time series encoder with attention, capable of learning transferable representations that perform competitively on various datasets, even with minimal adaptation.

Contribution

It presents a universal time series encoder that can be effectively adapted to new data types, outperforming existing methods in classification accuracy and efficiency.

Findings

Competitive accuracy on benchmark datasets

Effective with partial or no adaptation

Efficient and adaptable for scarce-labeled data

Abstract

We study the use of a time series encoder to learn representations that are useful on data set types with which it has not been trained on. The encoder is formed of a convolutional neural network whose temporal output is summarized by a convolutional attention mechanism. This way, we obtain a compact, fixed-length representation from longer, variable-length time series. We evaluate the performance of the proposed approach on a well-known time series classification benchmark, considering full adaptation, partial adaptation, and no adaptation of the encoder to the new data type. Results show that such strategies are competitive with the state-of-the-art, often outperforming conceptually-matching approaches. Besides accuracy scores, the facility of adaptation and the efficiency of pre-trained encoders make them an appealing option for the processing of scarcely- or non-labeled time series.