Can Multimodal LLMs Perform Time Series Anomaly Detection?

TL;DR

This paper explores the potential of multimodal large language models (MLLMs) for time series anomaly detection, introducing a comprehensive benchmark and a multi-agent framework to enhance detection capabilities in complex scenarios.

Contribution

It introduces the VisualTimeAnomaly benchmark for evaluating MLLMs on diverse TSAD tasks and proposes TSAD-Agents, a multi-agent system that improves automatic anomaly detection.

Findings

MLLMs show limited zero-shot performance on complex TSAD scenarios.

The VisualTimeAnomaly benchmark reveals challenges in detecting multi-granular and irregular anomalies.

The TSAD-Agents framework improves detection accuracy through adaptive multi-modal reasoning.

Abstract

Time series anomaly detection (TSAD) has been a long-standing pillar problem in Web-scale systems and online infrastructures, such as service reliability monitoring, system fault diagnosis, and performance optimization. Large language models (LLMs) have demonstrated unprecedented capabilities in time series analysis, the potential of multimodal LLMs (MLLMs), particularly vision-language models, in TSAD remains largely under-explored. One natural way for humans to detect time series anomalies is through visualization and textual description. It motivates our research question: Can multimodal LLMs perform time series anomaly detection? Existing studies often oversimplify the problem by treating point-wise anomalies as special cases of range-wise ones or by aggregating point anomalies to approximate range-wise scenarios. They limit our understanding for realistic scenarios such as multi-granular anomalies and irregular time series. To address the gap, we build a VisualTimeAnomaly benchmark to comprehensively investigate zero-shot capabilities of MLLMs for TSAD, progressively from point-, range-, to variate-wise anomalies, and extends to irregular sampling conditions. Our study reveals several key insights in multimodal MLLMs for TSAD. Built on these findings, we propose a MLLMs-based multi-agent framework TSAD-Agents to achieve automatic TSAD. Our framework comprises scanning, planning, detection, and checking agents that synergistically collaborate to reason, plan, and self-reflect to enable automatic TSAD. These agents adaptively invoke tools such as traditional methods and MLLMs and dynamically switch between text and image modalities to optimize detection performance.