Forecasting with Historical Data or Process Knowledge under Misspecification: A Comparison

TL;DR

This paper compares forecasting methods based on historical data versus process knowledge, showing that data-driven approaches often outperform knowledge-based ones under model misspecification, especially in complex systems.

Contribution

It provides a comprehensive simulation analysis demonstrating when historical data-based forecasting surpasses process knowledge methods under misspecification.

Findings

Forecasting with historical data is preferable under certain misspecification conditions.

Process knowledge-based forecasting can fail in highly nonlinear or chaotic systems.

Simulation results clarify scenarios favoring data-driven over knowledge-based forecasting.

Abstract

When faced with the task of forecasting a dynamic system, practitioners often have available historical data, knowledge of the system, or a combination of both. While intuition dictates that perfect knowledge of the system should in theory yield perfect forecasting, often knowledge of the system is only partially known, known up to parameters, or known incorrectly. In contrast, forecasting using previous data without any process knowledge might result in accurate prediction for simple systems, but will fail for highly nonlinear and chaotic systems. In this paper, the authors demonstrate how even in chaotic systems, forecasting with historical data is preferable to using process knowledge if this knowledge exhibits certain forms of misspecification. Through an extensive simulation study, a range of misspecification and forecasting scenarios are examined with the goal of gaining an improved understanding of the circumstances under which forecasting from historical data is to be preferred over using process knowledge.