In-context Time Series Predictor
Jiecheng Lu, Yan Sun, Shihao Yang
TL;DR
This paper introduces a novel in-context learning approach for time series forecasting using Transformers, reformulating the task as token-based input pairs to improve efficiency and performance without pre-trained models.
Contribution
It presents a new method that aligns time series forecasting with in-context learning, eliminating the need for pre-trained LLMs and reducing overfitting.
Findings
Outperforms previous Transformer-based TSF models in various settings
Achieves better results in full-data, few-shot, and zero-shot scenarios
Addresses overfitting issues in existing models
Abstract
Recent Transformer-based large language models (LLMs) demonstrate in-context learning ability to perform various functions based solely on the provided context, without updating model parameters. To fully utilize the in-context capabilities in time series forecasting (TSF) problems, unlike previous Transformer-based or LLM-based time series forecasting methods, we reformulate "time series forecasting tasks" as input tokens by constructing a series of (lookback, future) pairs within the tokens. This method aligns more closely with the inherent in-context mechanisms, and is more parameter-efficient without the need of using pre-trained LLM parameters. Furthermore, it addresses issues such as overfitting in existing Transformer-based TSF models, consistently achieving better performance across full-data, few-shot, and zero-shot settings compared to previous architectures.
Peer Reviews
Decision·ICLR 2025 Poster
ICTSP model transforms time series forecasting tasks into input tokens, aligning closely with the inherent mechanisms of the Transformer model and efficiently leveraging its contextual learning capabilities.
1. The innovation appears limited. While the paper extensively discusses the approach from an in-context learning perspective, the methodology itself seems simplistic and more akin to a heuristic trick due to excessive textual explanation without solid theoretical support. 2. The paper does not address whether this model could be applied to other time series tasks, such as classification, interpolation, or anomaly detection. The utility of the model appears confined to the forecasting tasks, ne
1. The writing is clear and fluent. 2. This paper's motivation is relatively novel. It proposes utilizing the in-context learning capabilities of large language models (LLMs) for time series forecasting tasks. 3. The experiments are comprehensive, encompassing various experimental settings and datasets.
1. The formulation of TSF transformers is not sufficient. As far as I am concerned, there are some methods [1,2] that utilize patching embedding to form the input tokens. These methods cannot be simply categorized as Temporal-wise Transformer or Series-wise Transformer in Section 2.2 2. The baselines results for baselines are collected from source papers. However, this paper applies different input time series length from these papers, which may lead to an unfair comparison. 3. The details of
1. **Novel Framework**: The paper proposes an innovative approach to time series forecasting by adopting in-context learning, which could inspire further research and development in this field. 2. **Comprehensive Experiments**: The authors present extensive experimental results, including ablation studies, which provide a robust foundation for evaluating ICTSP’s effectiveness. 3. **Clarity of Presentation**: The paper is well-written and clearly presented, facilitating a good understanding of th
1. **Comparative Fairness**: The baselines use an input length of 512, while ICTSP utilizes an input length of 1440. This discrepancy could impact the fairness of the comparisons. 2. **In-context Example Selection**: ICTSP frames forecasting as an in-context learning task, but it uses adjacent historical data as context examples. This approach raises questions about whether the model truly learns from in-context examples or simply encodes the input tokens in a different manner. 3. **Model Reduct
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTime Series Analysis and Forecasting · Neural Networks and Applications