Forecasting with Hyper-Trees

TL;DR

Hyper-Trees introduce a novel hybrid framework combining decision trees and neural networks to learn time series model parameters as functions of features, enhancing forecasting capabilities beyond traditional methods.

Contribution

This paper presents Hyper-Trees, a new approach that integrates trees and neural networks to model time series parameters, bridging decision tree effectiveness with classical forecasting models.

Findings

Effective across various forecasting tasks

Induces a time series inductive bias into tree models

Extends tree-based modeling beyond traditional time series analysis

Abstract

We introduce Hyper-Trees as a novel framework for modeling time series data using gradient boosted trees. Unlike conventional tree-based approaches that forecast time series directly, Hyper-Trees learn the parameters of a target time series model, such as ARIMA or Exponential Smoothing, as functions of features. These parameters are then used by the target model to generate the final forecasts. Our framework combines the effectiveness of decision trees on tabular data with classical forecasting models, thereby inducing a time series inductive bias into tree-based models. To resolve the scaling limitations of boosted trees when estimating a high-dimensional set of target model parameters, we combine decision trees and neural networks within a unified framework. In this hybrid approach, the trees generate informative representations from the input features, which a shallow network then uses as input to learn the parameters of a time series model. With our research, we explore the effectiveness of Hyper-Trees across a range of forecasting tasks and extend tree-based modeling beyond its conventional use in time series analysis.