MixLinear: Extreme Low Resource Multivariate Time Series Forecasting with 0.1K Parameters

TL;DR

MixLinear is a highly efficient multivariate time series forecasting model that uses only 0.1K parameters, capturing both temporal and frequency domain features, suitable for resource-constrained devices.

Contribution

The paper introduces MixLinear, a novel ultra-lightweight model that reduces parameter count from O(n^2) to O(n), maintaining high accuracy in long-term time series forecasting.

Findings

Achieves comparable or better accuracy than state-of-the-art models.

Uses only 0.1K parameters, enabling deployment on low-resource devices.

Reduces computational complexity significantly.

Abstract

Recently, there has been a growing interest in Long-term Time Series Forecasting (LTSF), which involves predicting long-term future values by analyzing a large amount of historical time-series data to identify patterns and trends. There exist significant challenges in LTSF due to its complex temporal dependencies and high computational demands. Although Transformer-based models offer high forecasting accuracy, they are often too compute-intensive to be deployed on devices with hardware constraints. On the other hand, the linear models aim to reduce the computational overhead by employing either decomposition methods in the time domain or compact representations in the frequency domain. In this paper, we propose MixLinear, an ultra-lightweight multivariate time series forecasting model specifically designed for resource-constrained devices. MixLinear effectively captures both temporal and frequency domain features by modeling intra-segment and inter-segment variations in the time domain and extracting frequency variations from a low-dimensional latent space in the frequency domain. By reducing the parameter scale of a downsampled $n$-length input/output one-layer linear model from $O(n^2)$ to $O(n)$, MixLinear achieves efficient computation without sacrificing accuracy. Extensive evaluations with four benchmark datasets show that MixLinear attains forecasting performance comparable to, or surpassing, state-of-the-art models with significantly fewer parameters ($0.1K$), which makes it well-suited for deployment on devices with limited computational capacity.