Mixture of Online and Offline Experts for Non-stationary Time Series

TL;DR

This paper introduces MOOE, a method that combines offline learned experts and an online expert to adaptively predict non-stationary time series, with theoretical guarantees on its performance.

Contribution

It proposes a novel framework that adaptively integrates offline and online experts for non-stationary time series prediction, supported by theoretical analysis.

Findings

Derived convergence, regret, and generalization bounds for MOOE.

Proved the effectiveness of combining offline and online experts.

Theoretically validated the adaptability of the proposed method.

Abstract

We consider a general and realistic scenario involving non-stationary time series, consisting of several offline intervals with different distributions within a fixed offline time horizon, and an online interval that continuously receives new samples. For non-stationary time series, the data distribution in the current online interval may have appeared in previous offline intervals. We theoretically explore the feasibility of applying knowledge from offline intervals to the current online interval. To this end, we propose the Mixture of Online and Offline Experts (MOOE). MOOE learns static offline experts from offline intervals and maintains a dynamic online expert for the current online interval. It then adaptively combines the offline and online experts using a meta expert to make predictions for the samples received in the online interval. Specifically, we focus on theoretical analysis, deriving parameter convergence, regret bounds, and generalization error bounds to prove the effectiveness of the algorithm.