Reasoning Pattern Alignment Merging for Adaptive Reasoning
Zhaofeng Zhong, Wei Yuan, Tong Chen, Xiangyu Zhao, Quoc Viet Hung Nguyen, Hongzhi Yin

TL;DR
This paper introduces RPAM, a layer-wise model merging framework that adaptively combines long and short chain-of-thought reasoning models, reducing inference costs while maintaining performance on reasoning tasks.
Contribution
The paper proposes a novel, training-free model merging method based on feature alignment for adaptive reasoning, improving efficiency in large reasoning models.
Findings
RPAM significantly reduces inference cost.
RPAM maintains strong reasoning performance.
RPAM adapts reasoning patterns effectively.
Abstract
Recent large reasoning models (LRMs) have made substantial progress in complex reasoning tasks, yet they often generate lengthy reasoning paths for every query, incurring unnecessary computation and latency. Existing speed-up approaches typically rely on retraining the model or designing sophisticated prompting, which are either prohibitively expensive or highly sensitive to the input and prompt formulation. In this work, we study model merging as a lightweight alternative for efficient reasoning: by combining a long chain-of-thought (Long-CoT) reasoning model with a Short-CoT instruction model, we obtain an adaptive reasoner without training from scratch or requiring large-scale additional data. Building on this idea, we propose Reasoning Pattern Alignment Merging (RPAM), a layer-wise model merging framework based on feature alignment to facilitate query-adaptive reasoning. RPAM first…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsConstraint Satisfaction and Optimization · AI-based Problem Solving and Planning · Advanced Graph Neural Networks
