Enhancing Transformer Efficiency for Multivariate Time Series Classification

TL;DR

This paper introduces a module-wise pruning and Pareto analysis approach to improve the efficiency of Transformer models for multivariate time series classification, balancing accuracy with computational costs.

Contribution

It presents a novel methodology combining pruning and Pareto analysis to optimize Transformer efficiency specifically for large-scale multivariate time series classification.

Findings

Effective reduction in training time and memory usage.

Maintains high classification accuracy after pruning.

Demonstrated on benchmark datasets.

Abstract

Most current multivariate time series (MTS) classification algorithms focus on improving the predictive accuracy. However, for large-scale (either high-dimensional or long-sequential) time series (TS) datasets, there is an additional consideration: to design an efficient network architecture to reduce computational costs such as training time and memory footprint. In this work we propose a methodology based on module-wise pruning and Pareto analysis to investigate the relationship between model efficiency and accuracy, as well as its complexity. Comprehensive experiments on benchmark MTS datasets illustrate the effectiveness of our method.