TempoGPT: Enhancing Time Series Reasoning via Quantizing Embedding

TL;DR

TempoGPT introduces a novel multi-modal time series language model that employs quantized embeddings to improve reasoning and alignment between temporal and textual data, achieving state-of-the-art results in complex reasoning tasks.

Contribution

The paper proposes TempoGPT, a multi-modal time series language model that uses quantized temporal embeddings for better reasoning and multi-modal alignment, addressing previous coarse labels and inconsistent representations.

Findings

TempoGPT achieves state-of-the-art performance in complex reasoning tasks.

Quantizing temporal embeddings enhances multi-modal alignment.

The approach improves the reasoning capabilities of multi-modal language models.

Abstract

Multi-modal language model has made advanced progress in vision and audio, but still faces significant challenges in dealing with complex reasoning tasks in the time series domain. The reasons are twofold. First, labels for multi-modal time series data are coarse and devoid of analysis or reasoning processes. Training with these data cannot improve the model's reasoning capabilities. Second, due to the lack of precise tokenization in processing time series, the representation patterns for temporal and textual information are inconsistent, which hampers the effectiveness of multi-modal alignment. To address these challenges, we propose a multi-modal time series data construction approach and a multi-modal time series language model (TLM), TempoGPT. Specially, we construct multi-modal data for complex reasoning tasks by analyzing the variable-system relationships within a white-box system. Additionally, proposed TempoGPT achieves consistent representation between temporal and textual information by quantizing temporal embeddings, where temporal embeddings are quantized into a series of discrete tokens using a predefined codebook; subsequently, a shared embedding layer processes both temporal and textual tokens. Extensive experiments demonstrate that TempoGPT accurately perceives temporal information, logically infers conclusions, and achieves state-of-the-art in the constructed complex time series reasoning tasks. Moreover, we quantitatively demonstrate the effectiveness of quantizing temporal embeddings in enhancing multi-modal alignment and the reasoning capabilities of TLMs. Code and data are available at https://github.com/zhanghaochuan20/TempoGPT.