TOKON: TOKenization-Optimized Normalization for time series analysis with a large language model

TL;DR

This paper introduces TOKON, a normalization technique that simplifies time series data for large language models by token reduction, and a new forecasting prompt, TFSC, both improving multi-step forecasting accuracy.

Contribution

The paper presents TOKON, a novel tokenization-aware normalization method, and TFSC, a specialized prompt, enhancing time series analysis with large language models.

Findings

TOKON reduces tokens by 2 to 3 times, improving model efficiency.

TOKON improves RMSE by 7% to 18% in multi-step forecasting.

TFSC further enhances forecasting accuracy when combined with TOKON.

Abstract

While large language models have rapidly evolved towards general artificial intelligence, their versatility in analyzing time series data remains limited. To address this limitation, we propose a novel normalization technique that considers the inherent nature of tokenization. The proposed Tokenization-Optimized Normalization (TOKON) simplifies time series data by representing each element with a single token, effectively reducing the number of tokens by 2 to 3 times. Additionally, we introduce a novel prompt for time series forecasting, termed Time Series Forecasting with Care (TFSC), to further enhance forecasting performance. Experimental results demonstrate that TOKON improves root mean square error (RMSE) for multi-step forecasting by approximately 7% to 18%, depending on the dataset and prompting method. Furthermore, TFSC, when used in conjunction with TOKON, shows additional improvements in forecasting accuracy for certain datasets