Hierarchical Multimodal LLMs with Semantic Space Alignment for Enhanced Time Series Classification

TL;DR

This paper introduces HiTime, a hierarchical multimodal LLM framework that aligns semantic spaces to improve time series classification by bridging the gap between numerical data and linguistic semantics, achieving superior results.

Contribution

The paper presents a novel hierarchical multimodal LLM approach with semantic space alignment and a parameter-efficient fine-tuning strategy for enhanced time series classification.

Findings

Outperforms state-of-the-art baselines on multiple benchmarks

Effectively bridges the representation gap between time series and semantics

Demonstrates the viability of generative LLMs for time series tasks

Abstract

Time series classification plays a fundamental role in a wide range of real-world applications. Recently, large language models (LLMs) have demonstrated strong generalization and reasoning capacities, but directly applying them to time series classification remains non-trivial due to the representation gap between numerical sequences and linguistic semantics. In this paper, we propose HiTime, a hierarchical LLM-based framework for multimodal time series classification that bridges structured temporal representations with semantic reasoning in a generative paradigm. Specifically, we design a hierarchical sequence feature encoding module composed of a data-specific encoder and a task-specific encoder to extract complementary temporal features. To mitigate the embedding gap between time series representations and textual semantics, we further introduce a semantic space alignment module that jointly performs coarse-grained global modeling and fine-grained cross-modal correspondence. Building upon the above representations, we employ a parameter-efficient supervised fine-tuning strategy to activate the generative classification capability of the algined LLMs, thereby transforming conventional discriminative time series classification into a generative task. Extensive experiments on multiple benchmarks demonstrate that the proposed framework consistently outperforms state-of-the-art baselines. The code is publicly available at https://github.com/Xiaoyu-Tao/HiTime.