Think Only When You Need with Large Hybrid-Reasoning Models

TL;DR

This paper introduces Large Hybrid-Reasoning Models (LHRMs) that adaptively decide when to perform extended reasoning, improving efficiency and reasoning capabilities over traditional models by combining hybrid training and reinforcement learning.

Contribution

The work presents the first adaptive hybrid reasoning model with a novel training pipeline and a metric for assessing hybrid thinking, enhancing reasoning efficiency and flexibility.

Findings

LHRMs outperform existing models in reasoning accuracy and general capabilities.

LHRMs significantly reduce token consumption and latency for simple queries.

The hybrid thinking metric effectively measures the model's reasoning adaptability.

Abstract

Recent Large Reasoning Models (LRMs) have shown substantially improved reasoning capabilities over traditional Large Language Models (LLMs) by incorporating extended thinking processes prior to producing final responses. However, excessively lengthy thinking introduces substantial overhead in terms of token consumption and latency, which is particularly unnecessary for simple queries. In this work, we introduce Large Hybrid-Reasoning Models (LHRMs), the first kind of model capable of adaptively determining whether to perform thinking based on the contextual information of user queries. To achieve this, we propose a two-stage training pipeline comprising Hybrid Fine-Tuning (HFT) as a cold start, followed by online reinforcement learning with the proposed Hybrid Group Policy Optimization (HGPO) to implicitly learn to select the appropriate thinking mode. Furthermore, we introduce a metric called Hybrid Accuracy to quantitatively assess the model's capability for hybrid thinking. Extensive experimental results show that LHRMs can adaptively perform hybrid thinking on queries of varying difficulty and type. It outperforms existing LRMs and LLMs in reasoning and general capabilities while significantly improving efficiency. Together, our work advocates for a reconsideration of the appropriate use of extended thinking processes and provides a solid starting point for building hybrid thinking systems.