MODE: Efficient Time Series Prediction with Mamba Enhanced by Low-Rank Neural ODEs

TL;DR

This paper introduces MODE, a novel framework combining Low-Rank Neural ODEs with an enhanced Mamba architecture to improve efficiency, scalability, and accuracy in long-range time series prediction across various domains.

Contribution

The paper presents a unified architecture integrating Low-Rank Neural ODEs with Mamba, introducing a segmented selective scanning mechanism for better long-term modeling and efficiency.

Findings

Outperforms existing methods in accuracy on benchmark datasets.

Reduces computational overhead via low-rank approximation.

Enhances long-range dependency modeling with selective scanning.

Abstract

Time series prediction plays a pivotal role across diverse domains such as finance, healthcare, energy systems, and environmental modeling. However, existing approaches often struggle to balance efficiency, scalability, and accuracy, particularly when handling long-range dependencies and irregularly sampled data. To address these challenges, we propose MODE, a unified framework that integrates Low-Rank Neural Ordinary Differential Equations (Neural ODEs) with an Enhanced Mamba architecture. As illustrated in our framework, the input sequence is first transformed by a Linear Tokenization Layer and then processed through multiple Mamba Encoder blocks, each equipped with an Enhanced Mamba Layer that employs Causal Convolution, SiLU activation, and a Low-Rank Neural ODE enhancement to efficiently capture temporal dynamics. This low-rank formulation reduces computational overhead while maintaining expressive power. Furthermore, a segmented selective scanning mechanism, inspired by pseudo-ODE dynamics, adaptively focuses on salient subsequences to improve scalability and long-range sequence modeling. Extensive experiments on benchmark datasets demonstrate that MODE surpasses existing baselines in both predictive accuracy and computational efficiency. Overall, our contributions include: (1) a unified and efficient architecture for long-term time series modeling, (2) integration of Mamba's selective scanning with low-rank Neural ODEs for enhanced temporal representation, and (3) substantial improvements in efficiency and scalability enabled by low-rank approximation and dynamic selective scanning.