MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models

TL;DR

MoFE-Time introduces a novel mixture of frequency and time domain experts within a pretraining-finetuning framework, significantly improving time series forecasting accuracy across multiple benchmarks and real-world datasets.

Contribution

The paper presents MoFE-Time, a new model that combines time and frequency domain features using a Mixture of Experts architecture with pretraining-finetuning, enhancing complex time series prediction.

Findings

Achieved state-of-the-art results on six public benchmarks.

Reduced MSE and MAE by approximately 7% and 6%.

Performed well on a proprietary real-world dataset.

Abstract

As a prominent data modality task, time series forecasting plays a pivotal role in diverse applications. With the remarkable advancements in Large Language Models (LLMs), the adoption of LLMs as the foundational architecture for time series modeling has gained significant attention. Although existing models achieve some success, they rarely both model time and frequency characteristics in a pretraining-finetuning paradigm leading to suboptimal performance in predictions of complex time series, which requires both modeling periodicity and prior pattern knowledge of signals. We propose MoFE-Time, an innovative time series forecasting model that integrates time and frequency domain features within a Mixture of Experts (MoE) network. Moreover, we use the pretraining-finetuning paradigm as our training framework to effectively transfer prior pattern knowledge across pretraining and finetuning datasets with different periodicity distributions. Our method introduces both frequency and time cells as experts after attention modules and leverages the MoE routing mechanism to construct multidimensional sparse representations of input signals. In experiments on six public benchmarks, MoFE-Time has achieved new state-of-the-art performance, reducing MSE and MAE by 6.95% and 6.02% compared to the representative methods Time-MoE. Beyond the existing evaluation benchmarks, we have developed a proprietary dataset, NEV-sales, derived from real-world business scenarios. Our method achieves outstanding results on this dataset, underscoring the effectiveness of the MoFE-Time model in practical commercial applications.