Deep TPC: Temporal-Prior Conditioning for Time Series Forecasting

TL;DR

Deep TPC introduces a novel approach to time series forecasting by elevating temporal information to a first-class modality, enabling multi-layer conditioning and disentangling temporal signals for improved long-term predictions.

Contribution

It proposes Temporal-Prior Conditioning (TPC), a method that conditions models at multiple depths using learnable tokens and temporal embeddings, outperforming existing strategies.

Findings

Achieves state-of-the-art long-term forecasting performance.

Outperforms full fine-tuning and shallow conditioning methods.

Maintains low parameter budget while enhancing temporal reasoning.

Abstract

LLM-for-time series (TS) methods typically treat time shallowly, injecting positional or prompt-based cues once at the input of a largely frozen decoder, which limits temporal reasoning as this information degrades through the layers. We introduce Temporal-Prior Conditioning (TPC), which elevates time to a first-class modality that conditions the model at multiple depths. TPC attaches a small set of learnable time series tokens to the patch stream; at selected layers these tokens cross-attend to temporal embeddings derived from compact, human-readable temporal descriptors encoded by the same frozen LLM, then feed temporal context back via self-attention. This disentangles time series signal and temporal information while maintaining a low parameter budget. We show that by training only the cross-attention modules and explicitly disentangling time series signal and temporal information, TPC consistently outperforms both full fine-tuning and shallow conditioning strategies, achieving state-of-the-art performance in long-term forecasting across diverse datasets. Code available at: https://github.com/fil-mp/Deep_tpc