Large Language Models are Few-shot Multivariate Time Series Classifiers

TL;DR

This paper introduces LLMFew, a novel framework leveraging large language models for few-shot multivariate time series classification, demonstrating significant accuracy improvements and potential for industrial applications with limited data.

Contribution

The paper proposes LLMFew, combining a Patch-wise Temporal Convolution Encoder and LoRA fine-tuning, to enhance LLMs for few-shot multivariate time series classification, a novel approach in this domain.

Findings

Outperformed state-of-the-art baselines by 125.2% and 50.2% in accuracy.

Demonstrated reliable performance across various datasets in few-shot settings.

Showed potential for industrial deployment with limited data.

Abstract

Large Language Models (LLMs) have been extensively applied in time series analysis. Yet, their utility in the few-shot classification (i.e., a crucial training scenario due to the limited training data available in industrial applications) concerning multivariate time series data remains underexplored. We aim to leverage the extensive pre-trained knowledge in LLMs to overcome the data scarcity problem within multivariate time series. Specifically, we propose LLMFew, an LLM-enhanced framework to investigate the feasibility and capacity of LLMs for few-shot multivariate time series classification. This model introduces a Patch-wise Temporal Convolution Encoder (PTCEnc) to align time series data with the textual embedding input of LLMs. We further fine-tune the pre-trained LLM decoder with Low-rank Adaptations (LoRA) to enhance its feature representation learning ability in time series data. Experimental results show that our model outperformed state-of-the-art baselines by a large margin, achieving 125.2% and 50.2% improvement in classification accuracy on Handwriting and EthanolConcentration datasets, respectively. Moreover, our experimental results demonstrate that LLM-based methods perform well across a variety of datasets in few-shot MTSC, delivering reliable results compared to traditional models. This success paves the way for their deployment in industrial environments where data are limited.