Dynamic Tokenization via Reinforcement Patching: End-to-end Training and Zero-shot Transfer

TL;DR

ReinPatch introduces a reinforcement learning framework for dynamic, data-driven sequence patching that jointly optimizes patch boundaries and model performance, enhancing long-horizon sequence modeling.

Contribution

It is the first method to jointly optimize sequence patching policies and models end-to-end using reinforcement learning, enabling adaptive, hierarchical, and rate-controlled sequence representations.

Findings

ReinPatch outperforms state-of-the-art patching strategies on time-series forecasting.

The framework allows explicit control over compression rates.

The patching module can be extracted as a standalone component for analysis.

Abstract

Efficiently aggregating spatial or temporal horizons to acquire compact representations has become a unifying principle in modern deep learning models, yet learning data-adaptive representations for long-horizon sequence data, especially continuous sequences like time series, remains an open challenge. While fixed-size patching has improved scalability and performance, discovering variable-sized, data-driven patches end-to-end often forces models to rely on soft discretization, specific backbones, or heuristic rules. In this work, we propose Reinforcement Patching (ReinPatch), the first framework to jointly optimize a sequence patching policy and its downstream sequence backbone model using reinforcement learning. By formulating patch boundary placement as a discrete decision process optimized via Group Relative Policy Gradient (GRPG), ReinPatch bypasses the need for continuous relaxations and performs dynamic patching policy optimization in a natural manner. Moreover, our method allows strict enforcement of a desired compression rate, freeing the downstream backbone to scale efficiently, and naturally supports multi-level hierarchical modeling. We evaluate ReinPatch on time-series forecasting datasets, where it demonstrates compelling performance compared to state-of-the-art data-driven patching strategies. Furthermore, our detached design allows the patching module to be extracted as a standalone foundation patcher, providing the community with visual and empirical insights into the segmentation behaviors preferred by a purely performance-driven neural patching strategy.