BoundRL: Efficient Structured Text Segmentation through Reinforced Boundary Generation

TL;DR

BoundRL introduces an efficient token-level segmentation method for structured texts, leveraging reinforcement learning to improve boundary detection and outperform larger models in complex prompt tasks.

Contribution

It proposes BoundRL, a novel reinforcement learning approach that reduces token output and hallucination in structured text segmentation, enabling smaller models to excel.

Findings

Reduces output tokens by 90% compared to full text generation.

Enables small models to outperform larger models in complex prompt tasks.

Uses reinforcement learning with verifiable rewards for boundary generation.

Abstract

Structured texts refer to texts containing structured elements beyond plain texts, such as code snippets and placeholders. Such structured texts increasingly require segmentation into semantically meaningful components, which cannot be effectively handled by conventional sentence-level segmentation methods. To address this, we propose BoundRL, a novel approach that jointly performs efficient token-level text segmentation and label prediction for long structured texts. Instead of generating full texts for each segment, it generates only starting tokens and reconstructs the complete texts by locating these tokens within the original texts, thereby reducing output tokens by 90% and minimizing hallucination. To train the models for the boundary generation, BoundRL~performs reinforcement learning with verifiable rewards (RLVR) that jointly optimizes document reconstruction fidelity and semantic alignment. It further mitigates entropy collapse by constructing intermediate candidates by perturbing segment boundaries and labels to create stepping stones toward higher-quality solutions. Experiments show that BoundRL enables small language models (1.7B parameters) to outperform few-shot prompting with much larger models as well as SFT and standard RLVR baselines on complex prompts used for LLM applications.