TimeSqueeze: Dynamic Patching for Efficient Time Series Forecasting

TL;DR

TimeSqueeze introduces a dynamic patching method for time series transformers that adaptively segments data based on local complexity, significantly improving efficiency and forecasting accuracy.

Contribution

It proposes a novel content-aware segmentation mechanism that adaptively chooses patch boundaries, enhancing temporal fidelity and computational efficiency in transformer-based models.

Findings

Up to 20x faster convergence in large-scale pretraining

8x higher data efficiency compared to point-token baselines

Consistently outperforms fixed patching methods on forecasting benchmarks

Abstract

Transformer-based time series foundation models face a fundamental trade-off in choice of tokenization: point-wise embeddings preserve temporal fidelity but scale poorly with sequence length, whereas fixed-length patching improves efficiency by imposing uniform boundaries that may disrupt natural transitions and blur informative local dynamics. In order to address these limitations, we introduce TimeSqueeze, a dynamic patching mechanism that adaptively selects patch boundaries within each sequence based on local signal complexity. TimeSqueeze first applies a lightweight state-space encoder to extract full-resolution point-wise features, then performs content-aware segmentation by allocating short patches to information-dense regions and long patches to smooth or redundant segments. This variable-resolution compression preserves critical temporal structure while substantially reducing the token sequence presented to the Transformer backbone. Specifically for large-scale pretraining, TimeSqueeze attains up to 20x faster convergence and 8x higher data efficiency compared to equivalent point-token baselines. Experiments across long-horizon forecasting benchmarks show that TimeSqueeze consistently outperforms comparable architectures that use either point-wise tokenization or fixed-size patching.