CMamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecasting

TL;DR

CMamba is a novel multivariate time series forecasting model that enhances cross-channel dependency capture using a global data-dependent MLP and a channel mixup mechanism, outperforming existing models.

Contribution

The paper introduces CMamba, a refined Mamba variant with a global data-dependent MLP and channel mixup, specifically designed to improve cross-channel dependency modeling in time series forecasting.

Findings

CMamba outperforms baseline models on seven real-world datasets.

The global data-dependent MLP improves cross-channel dependency capture.

Channel Mixup reduces overfitting and enhances generalization.

Abstract

Recent advancements in multivariate time series forecasting have been propelled by Linear-based, Transformer-based, and Convolution-based models, with Transformer-based architectures gaining prominence for their efficacy in temporal and cross-channel mixing. More recently, Mamba, a state space model, has emerged with robust sequence and feature mixing capabilities. However, the suitability of the vanilla Mamba design for time series forecasting remains an open question, particularly due to its inadequate handling of cross-channel dependencies. Capturing cross-channel dependencies is critical in enhancing the performance of multivariate time series prediction. Recent findings show that self-attention excels in capturing cross-channel dependencies, whereas other simpler mechanisms, such as MLP, may degrade model performance. This is counterintuitive, as MLP, being a learnable architecture, should theoretically capture both correlations and irrelevances, potentially leading to neutral or improved performance. Diving into the self-attention mechanism, we attribute the observed degradation in MLP performance to its lack of data dependence and global receptive field, which result in MLP's lack of generalization ability. Based on the above insights, we introduce a refined Mamba variant tailored for time series forecasting. Our proposed model, \textbf{CMamba}, incorporates a modified Mamba (M-Mamba) module for temporal dependencies modeling, a global data-dependent MLP (GDD-MLP) to effectively capture cross-channel dependencies, and a Channel Mixup mechanism to mitigate overfitting. Comprehensive experiments conducted on seven real-world datasets demonstrate the efficacy of our model in improving forecasting performance.