Recurrent Neural Goodness-of-Fit Test for Time Series

TL;DR

This paper introduces RENAL, a new goodness-of-fit test for time series that uses recurrent neural networks to evaluate generative models' performance, addressing limitations of existing metrics.

Contribution

The paper proposes a statistically rigorous, neural network-based framework for evaluating generative time series models, capable of handling dependencies and high-dimensional data.

Findings

Outperforms existing evaluation methods in synthetic datasets

Effective in real-world healthcare and finance data

Robustly assesses model quality with limited data

Abstract

Time series data are crucial across diverse domains such as finance and healthcare, where accurate forecasting and decision-making rely on advanced modeling techniques. While generative models have shown great promise in capturing the intricate dynamics inherent in time series, evaluating their performance remains a major challenge. Traditional evaluation metrics fall short due to the temporal dependencies and potential high dimensionality of the features. In this paper, we propose the REcurrent NeurAL (RENAL) Goodness-of-Fit test, a novel and statistically rigorous framework for evaluating generative time series models. By leveraging recurrent neural networks, we transform the time series into conditionally independent data pairs, enabling the application of a chi-square-based goodness-of-fit test to the temporal dependencies within the data. This approach offers a robust, theoretically grounded solution for assessing the quality of generative models, particularly in settings with limited time sequences. We demonstrate the efficacy of our method across both synthetic and real-world datasets, outperforming existing methods in terms of reliability and accuracy. Our method fills a critical gap in the evaluation of time series generative models, offering a tool that is both practical and adaptable to high-stakes applications.