Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervision

TL;DR

This paper investigates the effectiveness of Chain-of-Thought prompting for long-context reasoning in large language models and introduces LongRePS, a framework that enhances reasoning path quality to improve long-context task performance.

Contribution

The paper demonstrates that Chain-of-Thought benefits extend to long-context tasks and introduces LongRePS, a process-supervised method for generating high-quality reasoning paths in such scenarios.

Findings

CoT benefits generalize to long-context tasks and increase with context length.

LongRePS significantly improves performance on long-context benchmarks.

The approach achieves notable gains over outcome supervision baselines.

Abstract

Recent advances in Large Language Models (LLMs) have highlighted the challenge of handling long-context tasks, where models need to reason over extensive input contexts to aggregate target information. While Chain-of-Thought (CoT) prompting has shown promise for multi-step reasoning, its effectiveness for long-context scenarios remains underexplored. Through systematic investigation across diverse tasks, we demonstrate that CoT's benefits generalize across most long-context scenarios and amplify with increasing context length. Motivated by this critical observation, we propose LongRePS, a process-supervised framework that teaches models to generate high-quality reasoning paths for enhanced long-context performance. Our framework incorporates a self-sampling mechanism to bootstrap reasoning paths and a novel quality assessment protocol specifically designed for long-context scenarios. Experimental results on various long-context benchmarks demonstrate the effectiveness of our approach, achieving significant improvements over outcome supervision baselines on both in-domain tasks (+13.6/+3.8 points for LLaMA/Qwen on MuSiQue) and cross-domain generalization (+9.3/+8.1 points on average across diverse QA tasks). Our code, data and trained models are made public to facilitate future research.