Time Series Language Model for Descriptive Caption Generation

TL;DR

This paper introduces TSLM, a novel encoder-decoder language model designed for generating descriptive captions of time series data, addressing data scarcity and capturing subtle temporal patterns for improved interpretability.

Contribution

TSLM is the first specialized time series captioning model leveraging in-context synthetic data generation and cross-modal dense retrieval to enhance caption quality.

Findings

TSLM outperforms existing methods on multiple datasets.

Synthetic data generation improves model training.

Cross-modal dense retrieval enhances caption accuracy.

Abstract

The automatic generation of representative natural language descriptions for observable patterns in time series data enhances interpretability, simplifies analysis and increases cross-domain utility of temporal data. While pre-trained foundation models have made considerable progress in natural language processing (NLP) and computer vision (CV), their application to time series analysis has been hindered by data scarcity. Although several large language model (LLM)-based methods have been proposed for time series forecasting, time series captioning is under-explored in the context of LLMs. In this paper, we introduce TSLM, a novel time series language model designed specifically for time series captioning. TSLM operates as an encoder-decoder model, leveraging both text prompts and time series data representations to capture subtle temporal patterns across multiple phases and generate precise textual descriptions of time series inputs. TSLM addresses the data scarcity problem in time series captioning by first leveraging an in-context prompting synthetic data generation, and second denoising the generated data via a novel cross-modal dense retrieval scoring applied to time series-caption pairs. Experimental findings on various time series captioning datasets demonstrate that TSLM outperforms existing state-of-the-art approaches from multiple data modalities by a significant margin.