Class-Specific Attention (CSA) for Time-Series Classification

TL;DR

This paper introduces a class-specific attention (CSA) module that enhances neural network models for time-series classification by focusing on class-distinctive features, leading to significant accuracy improvements across multiple datasets.

Contribution

The paper proposes a novel CSA module that can be integrated into existing neural networks to better capture class-specific features in time-series data.

Findings

CSA improves accuracy up to 42% in some cases.

Embedding CSA outperforms base models on 67% of MTS and 80% of UTS datasets.

Performance is significantly better in 11% of MTS and 13% of UTS cases.

Abstract

Most neural network-based classifiers extract features using several hidden layers and make predictions at the output layer by utilizing these extracted features. We observe that not all features are equally pronounced in all classes; we call such features class-specific features. Existing models do not fully utilize the class-specific differences in features as they feed all extracted features from the hidden layers equally to the output layers. Recent attention mechanisms allow giving different emphasis (or attention) to different features, but these attention models are themselves class-agnostic. In this paper, we propose a novel class-specific attention (CSA) module to capture significant class-specific features and improve the overall classification performance of time series. The CSA module is designed in a way such that it can be adopted in existing neural network (NN) based models to conduct time series classification. In the experiments, this module is plugged into five start-of-the-art neural network models for time series classification to test its effectiveness by using 40 different real datasets. Extensive experiments show that an NN model embedded with the CSA module can improve the base model in most cases and the accuracy improvement can be up to 42%. Our statistical analysis show that the performance of an NN model embedding the CSA module is better than the base NN model on 67% of MTS and 80% of UTS test cases and is significantly better on 11% of MTS and 13% of UTS test cases.