TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series

TL;DR

This paper introduces TEST, a novel method to enable large language models to effectively handle time series data by embedding and aligning time series representations with LLMs, improving performance on various TS tasks.

Contribution

The paper proposes TEST, a new TS embedding and alignment method that activates LLMs for time series tasks without retraining, demonstrating competitive results across multiple benchmarks.

Findings

TEST achieves state-of-the-art or comparable performance on TS classification and forecasting.

The method enhances few-shot learning and generalization capabilities of LLMs for TS data.

Frozen LLMs with TEST outperform traditional TS models in experiments.

Abstract

This work summarizes two ways to accomplish Time-Series (TS) tasks in today's Large Language Model (LLM) context: LLM-for-TS (model-centric) designs and trains a fundamental large model, or fine-tunes a pre-trained LLM for TS data; TS-for-LLM (data-centric) converts TS into a model-friendly representation to enable the pre-trained LLM to handle TS data. Given the lack of data, limited resources, semantic context requirements, and so on, this work focuses on TS-for-LLM, where we aim to activate LLM's ability for TS data by designing a TS embedding method suitable for LLM. The proposed method is named TEST. It first tokenizes TS, builds an encoder to embed TS via instance-wise, feature-wise, and text-prototype-aligned contrast, where the TS embedding space is aligned to LLM embedding layer space, then creates soft prompts to make LLM more open to that embeddings, and finally implements TS tasks using the frozen LLM. We also demonstrate the feasibility of TS-for-LLM through theory and experiments. Experiments are carried out on TS classification, forecasting, and representation tasks using eight frozen LLMs with various structures and sizes. The results show that the pre-trained LLM with TEST strategy can achieve better or comparable performance than today's SOTA TS models and offer benefits for few-shot and generalization. By treating LLM as the pattern machine, TEST can endow LLM's ability to process TS data without compromising language ability. We hope that this study will serve as a foundation for future work to support TS+LLM progress.