Adversarial Vulnerabilities in Large Language Models for Time Series Forecasting

TL;DR

This paper reveals that large language models used for time series forecasting are highly vulnerable to adversarial attacks, significantly impairing their accuracy and highlighting the need for improved robustness.

Contribution

It introduces a novel targeted adversarial attack framework for LLM-based time series forecasting, demonstrating its effectiveness across multiple models and datasets.

Findings

Adversarial attacks cause severe performance degradation.

Attacks are effective across different LLM architectures.

Random noise has less impact than targeted adversarial perturbations.

Abstract

Large Language Models (LLMs) have recently demonstrated significant potential in time series forecasting, offering impressive capabilities in handling complex temporal data. However, their robustness and reliability in real-world applications remain under-explored, particularly concerning their susceptibility to adversarial attacks. In this paper, we introduce a targeted adversarial attack framework for LLM-based time series forecasting. By employing both gradient-free and black-box optimization methods, we generate minimal yet highly effective perturbations that significantly degrade the forecasting accuracy across multiple datasets and LLM architectures. Our experiments, which include models like LLMTime with GPT-3.5, GPT-4, LLaMa, and Mistral, TimeGPT, and TimeLLM show that adversarial attacks lead to much more severe performance degradation than random noise, and demonstrate the broad effectiveness of our attacks across different LLMs. The results underscore the critical vulnerabilities of LLMs in time series forecasting, highlighting the need for robust defense mechanisms to ensure their reliable deployment in practical applications. The code repository can be found at https://github.com/JohnsonJiang1996/AdvAttack_LLM4TS.