Causal Time-Series Synchronization for Multi-Dimensional Forecasting

TL;DR

This paper introduces a novel pre-training strategy for multi-dimensional time-series data in Digital Twins, leveraging synchronized cause-effect pairs to improve forecasting accuracy and generalization across diverse domains.

Contribution

It proposes a channel-dependent pre-training approach that identifies and utilizes lagged causal relationships in multi-dimensional time-series data for better forecasting.

Findings

Significant improvements in forecasting accuracy.

Enhanced generalization across different tasks and domains.

Effective identification of lagged causal relationships.

Abstract

The process industry's high expectations for Digital Twins require modeling approaches that can generalize across tasks and diverse domains with potentially different data dimensions and distributional shifts i.e., Foundational Models. Despite success in natural language processing and computer vision, transfer learning with (self-) supervised signals for pre-training general-purpose models is largely unexplored in the context of Digital Twins in the process industry due to challenges posed by multi-dimensional time-series data, lagged cause-effect dependencies, complex causal structures, and varying number of (exogenous) variables. We propose a novel channel-dependent pre-training strategy that leverages synchronized cause-effect pairs to overcome these challenges by breaking down the multi-dimensional time-series data into pairs of cause-effect variables. Our approach focuses on: (i) identifying highly lagged causal relationships using data-driven methods, (ii) synchronizing cause-effect pairs to generate training samples for channel-dependent pre-training, and (iii) evaluating the effectiveness of this approach in channel-dependent forecasting. Our experimental results demonstrate significant improvements in forecasting accuracy and generalization capability compared to traditional training methods.