Segment-Level Attribution for Selective Learning of Long Reasoning Traces

TL;DR

This paper introduces a segment-level attribution method using integrated gradients to identify and focus on reflective reasoning segments in long chains of thought, improving model accuracy and efficiency.

Contribution

It proposes a novel segment-level attribution framework with selective finetuning to enhance reasoning quality in large models.

Findings

Improves reasoning accuracy across multiple datasets.

Reduces unnecessary verbose content in generated traces.

Enhances learning efficiency by focusing on important segments.

Abstract

Large Reasoning Models (LRMs) achieve strong reasoning performance by generating long chains of thought (CoTs), yet only a small fraction of these traces meaningfully contributes to answer prediction, while the majority contains repetitive or truncated content. Such output redundancy is further propagated after supervised finetuning (SFT), as models learn to imitate verbose but uninformative patterns, which can degrade performance. To this end, we incorporate integrated gradient attribution to quantify each token's influence on final answers and aggregate them into two segment-level metrics: (1) \textit{attribution strength} measures the overall attribution magnitude; and (2) \textit{direction consistency} captures whether tokens' attributions within a segment are uniformly positive or negative (high consistency), or a mixture of both (moderate consistency). Based on these two metrics, we propose a segment-level selective learning framework to identify important segments with high attribution strength but moderate consistency that indicate reflective rather than shallow reasoning. The framework then applies selective SFT on these important segments while masking loss for unimportant ones. Experiments across multiple models and datasets show that our approach improves accuracy and output efficiency, enabling more effective learning from long reasoning traces~\footnote{Code and data are available at https://github.com/SiyuanWangw/SegmentSelectiveSFT}.