An Exploratory Study to Repurpose LLMs to a Unified Architecture for Time Series Classification

TL;DR

This paper explores hybrid architectures combining specialized time series encoders with frozen large language models, highlighting the significance of encoder choice, especially Inception, for effective time series classification.

Contribution

It systematically evaluates various encoder architectures in hybrid LLM models for TSC, revealing Inception as the most effective encoder architecture.

Findings

Inception encoder consistently improves performance with LLMs.

Encoder architecture choice significantly impacts hybrid TSC models.

Inception-based models are promising for future LLM-driven TSC.

Abstract

Time series classification (TSC) is a core machine learning problem with broad applications. Recently there has been growing interest in repurposing large language models (LLMs) for TSC, motivated by their strong reasoning and generalization ability. Prior work has primarily focused on alignment strategies that explicitly map time series data into the textual domain; however, the choice of time series encoder architecture remains underexplored. In this work, we conduct an exploratory study of hybrid architectures that combine specialized time series encoders with a frozen LLM backbone. We evaluate a diverse set of encoder families, including Inception, convolutional, residual, transformer-based, and multilayer perceptron architectures, among which the Inception model is the only encoder architecture that consistently yields positive performance gains when integrated with an LLM backbone. Overall, this study highlights the impact of time series encoder choice in hybrid LLM architectures and points to Inception-based models as a promising direction for future LLM-driven time series learning.