QFFT, Question-Free Fine-Tuning for Adaptive Reasoning

TL;DR

QFFT is a novel fine-tuning method that enables models to adaptively switch between concise and detailed reasoning patterns without requiring input questions, improving efficiency and robustness across tasks.

Contribution

The paper introduces Question-Free Fine-Tuning (QFFT), a new approach that trains models on reasoning responses alone, allowing adaptive reasoning pattern selection without question input.

Findings

QFFT reduces response length by over 50%.

QFFT maintains performance comparable to supervised fine-tuning.

QFFT outperforms SFT in noisy, out-of-domain, and low-resource settings.

Abstract

Recent advancements in Long Chain-of-Thought (CoT) reasoning models have improved performance on complex tasks, but they suffer from overthinking, which generates redundant reasoning steps, especially for simple questions. This paper revisits the reasoning patterns of Long and Short CoT models, observing that the Short CoT patterns offer concise reasoning efficiently, while the Long CoT patterns excel in challenging scenarios where the Short CoT patterns struggle. To enable models to leverage both patterns, we propose Question-Free Fine-Tuning (QFFT), a fine-tuning approach that removes the input question during training and learns exclusively from Long CoT responses. This approach enables the model to adaptively employ both reasoning patterns: it prioritizes the Short CoT patterns and activates the Long CoT patterns only when necessary. Experiments on various mathematical datasets demonstrate that QFFT reduces average response length by more than 50\%, while achieving performance comparable to Supervised Fine-Tuning (SFT). Additionally, QFFT exhibits superior performance compared to SFT in noisy, out-of-domain, and low-resource scenarios.