Time-Series Forecasting, Knowledge Distillation, and Refinement within a Multimodal PDE Foundation Model

TL;DR

This paper introduces an automated, low-cost symbolic encoding method using SymPy for multimodal time-series forecasting of PDE systems, enhancing accuracy through Bayesian refinement.

Contribution

It presents a novel SymPy-based token library for symbolic PDE encoding, enabling automated, flexible multimodal learning with improved predictive accuracy.

Findings

High prediction accuracy maintained with minimal symbolic preprocessing

Automated PDE encoding reduces manual effort and costs

Bayesian filtering refines symbolic and predictive models

Abstract

Symbolic encoding has been used in multi-operator learning as a way to embed additional information for distinct time-series data. For spatiotemporal systems described by time-dependent partial differential equations, the equation itself provides an additional modality to identify the system. The utilization of symbolic expressions along side time-series samples allows for the development of multimodal predictive neural networks. A key challenge with current approaches is that the symbolic information, i.e. the equations, must be manually preprocessed (simplified, rearranged, etc.) to match and relate to the existing token library, which increases costs and reduces flexibility, especially when dealing with new differential equations. We propose a new token library based on SymPy to encode differential equations as an additional modality for time-series models. The proposed approach incurs minimal cost, is automated, and maintains high prediction accuracy for forecasting tasks. Additionally, we include a Bayesian filtering module that connects the different modalities to refine the learned equation. This improves the accuracy of the learned symbolic representation and the predicted time-series.