Temporal Convolutional Explorer Helps Understand 1D-CNN's Learning Behavior in Time Series Classification from Frequency Domain

TL;DR

This paper introduces a method to analyze and improve 1D-CNNs for time series classification by examining their frequency domain learning behavior, leading to enhanced performance and efficiency.

Contribution

We propose the Temporal Convolutional Explorer (TCE) for empirical analysis and develop a regulatory framework to improve 1D-CNN learning by bypassing disturbing convolutions.

Findings

Deeper 1D-CNNs focus less on low-frequency components, causing accuracy issues.

TCE provides valuable insights into 1D-CNN learning behavior.

Our framework improves 1D-CNN performance with reduced memory and computational costs.

Abstract

While one-dimensional convolutional neural networks (1D-CNNs) have been empirically proven effective in time series classification tasks, we find that there remain undesirable outcomes that could arise in their application, motivating us to further investigate and understand their underlying mechanisms. In this work, we propose a Temporal Convolutional Explorer (TCE) to empirically explore the learning behavior of 1D-CNNs from the perspective of the frequency domain. Our TCE analysis highlights that deeper 1D-CNNs tend to distract the focus from the low-frequency components leading to the accuracy degradation phenomenon, and the disturbing convolution is the driving factor. Then, we leverage our findings to the practical application and propose a regulatory framework, which can easily be integrated into existing 1D-CNNs. It aims to rectify the suboptimal learning behavior by enabling the network to selectively bypass the specified disturbing convolutions. Finally, through comprehensive experiments on widely-used UCR, UEA, and UCI benchmarks, we demonstrate that 1) TCE's insight into 1D-CNN's learning behavior; 2) our regulatory framework enables state-of-the-art 1D-CNNs to get improved performances with less consumption of memory and computational overhead.