Multi-Faceted Representation Learning with Hybrid Architecture for Time Series Classification

TL;DR

This paper introduces SARCoN, a hybrid neural network architecture combining LSTM, self-attention, and convolutional modules to improve univariate time series classification accuracy and interpretability across diverse domains.

Contribution

The paper proposes a novel hybrid neural architecture, SARCoN, that learns multi-faceted representations for time series classification, demonstrating superior performance and interpretability over existing methods.

Findings

Achieves state-of-the-art results on UCR benchmarks.

Enables interpretability through self-attention and global pooling.

Generalizes well across diverse domain tasks.

Abstract

Time series classification problems exist in many fields and have been explored for a couple of decades. However, they still remain challenging, and their solutions need to be further improved for real-world applications in terms of both accuracy and efficiency. In this paper, we propose a hybrid neural architecture, called Self-Attentive Recurrent Convolutional Networks (SARCoN), to learn multi-faceted representations for univariate time series. SARCoN is the synthesis of long short-term memory networks with self-attentive mechanisms and Fully Convolutional Networks, which work in parallel to learn the representations of univariate time series from different perspectives. The component modules of the proposed architecture are trained jointly in an end-to-end manner and they classify the input time series in a cooperative way. Due to its domain-agnostic nature, SARCoN is able to generalize a diversity of domain tasks. Our experimental results show that, compared to the state-of-the-art approaches for time series classification, the proposed architecture can achieve remarkable improvements for a set of univariate time series benchmarks from the UCR repository. Moreover, the self-attention and the global average pooling in the proposed architecture enable visible interpretability by facilitating the identification of the contribution regions of the original time series. An overall analysis confirms that multi-faceted representations of time series aid in capturing deep temporal corrections within complex time series, which is essential for the improvement of time series classification performance. Our work provides a novel angle that deepens the understanding of time series classification, qualifying our proposed model as an ideal choice for real-world applications.