ConvTimeNet: A Deep Hierarchical Fully Convolutional Model for Multivariate Time Series Analysis

TL;DR

ConvTimeNet is a hierarchical convolutional model that adaptively captures local patterns and multi-scale dependencies in multivariate time series, outperforming existing methods without relying on self-attention.

Contribution

It introduces a deformable patch layer and large kernel mechanism within a pure convolutional framework for improved time series analysis.

Findings

Outperforms various models across multiple tasks

Effectively captures local and multi-scale dependencies

Demonstrates strong viability of pure convolutional models

Abstract

Designing effective models for learning time series representations is foundational for time series analysis. Many previous works have explored time series representation modeling approaches and have made progress in this area. Despite their effectiveness, they lack adaptive perception of local patterns in temporally dependent basic units and fail to capture the multi-scale dependency among these units. Instead of relying on prevalent methods centered around self-attention mechanisms, we propose ConvTimeNet, a hierarchical pure convolutional model designed for time series analysis. ConvTimeNet introduces a deformable patch layer that adaptively perceives local patterns of temporally dependent basic units in a data-driven manner. Based on the extracted local patterns, hierarchical pure convolutional blocks are designed to capture dependency relationships among the representations of basic units at different scales. Moreover, a large kernel mechanism is employed to ensure that convolutional blocks can be deeply stacked, thereby achieving a larger receptive field. In this way, local patterns and their multi-scale dependencies can be effectively modeled within a single model. Extensive experiments comparing a wide range of different types of models demonstrate that pure convolutional models still exhibit strong viability, effectively addressing the aforementioned two challenges and showing superior performance across multiple tasks. The code is available for reproducibility.