SEED: A Structural Encoder for Embedding-Driven Decoding in Time Series Prediction with LLMs

TL;DR

SEED introduces a novel structural encoder that combines feature interaction modeling with semantic reasoning, enabling improved multivariate time series forecasting by integrating LLMs with time series data.

Contribution

The paper presents SEED, a modular architecture that aligns time series features with language model embeddings for enhanced prediction and reasoning capabilities.

Findings

Achieves consistent improvements over strong baselines

Effectively bridges structural and semantic modeling gaps

Demonstrates versatility across various datasets

Abstract

Multivariate time series forecasting requires models to simultaneously capture variable-wise structural dependencies and generalize across diverse tasks. While structural encoders are effective in modeling feature interactions, they lack the capacity to support semantic-level reasoning or task adaptation. Conversely, large language models (LLMs) possess strong generalization capabilities but remain incompatible with raw time series inputs. This gap limits the development of unified, transferable prediction systems. Therefore, we introduce SEED, a structural encoder for embedding-driven decoding, which integrates four stages: a token-aware encoder for patch extraction, a projection module that aligns patches with language model embeddings, a semantic reprogramming mechanism that maps patches to task-aware prototypes, and a frozen language model for prediction. This modular architecture decouples representation learning from inference, enabling efficient alignment between numerical patterns and semantic reasoning. Empirical results demonstrate that the proposed method achieves consistent improvements over strong baselines, and comparative studies on various datasets confirm SEED's role in addressing the structural-semantic modeling gap.