N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecasting

TL;DR

N-BEATS-MOE enhances the original N-BEATS model by integrating a Mixture-of-Experts layer with a gating mechanism, improving adaptability and interpretability for heterogeneous time series forecasting across multiple datasets.

Contribution

This work introduces N-BEATS-MOE, a novel extension of N-BEATS that incorporates a Mixture-of-Experts layer with dynamic weighting and interpretability features.

Findings

Achieves consistent improvements on heterogeneous datasets

Demonstrates better adaptability to diverse time series

Provides insights into expert relevance for series

Abstract

Deep learning approaches are increasingly relevant for time series forecasting tasks. Methods such as N-BEATS, which is built on stacks of multilayer perceptrons (MLPs) blocks, have achieved state-of-the-art results on benchmark datasets and competitions. N-BEATS is also more interpretable relative to other deep learning approaches, as it decomposes forecasts into different time series components, such as trend and seasonality. In this work, we present N-BEATS-MOE, an extension of N-BEATS based on a Mixture-of-Experts (MoE) layer. N-BEATS-MOE employs a dynamic block weighting strategy based on a gating network which allows the model to better adapt to the characteristics of each time series. We also hypothesize that the gating mechanism provides additional interpretability by identifying which expert is most relevant for each series. We evaluate our method across 12 benchmark datasets against several approaches, achieving consistent improvements on several datasets, especially those composed of heterogeneous time series.