Ada-RS: Adaptive Rejection Sampling for Selective Thinking

TL;DR

Ada-RS is a versatile sampling framework that enhances the efficiency of large language models by selectively filtering reasoning steps, significantly reducing token usage and reasoning time without sacrificing accuracy.

Contribution

Introduces Ada-RS, an adaptive rejection sampling method that improves reasoning efficiency in tool-using LLMs by selective filtering of generated samples.

Findings

Reduces average output tokens by up to 80%.

Decreases thinking rate by up to 95%.

Maintains or improves tool call accuracy.

Abstract

Large language models (LLMs) are increasingly being deployed in cost and latency-sensitive settings. While chain-of-thought improves reasoning, it can waste tokens on simple requests. We study selective thinking for tool-using LLMs and introduce Adaptive Rejection Sampling (Ada-RS), an algorithm-agnostic sample filtering framework for learning selective and efficient reasoning. For each given context, Ada-RS scores multiple sampled completions with an adaptive length-penalized reward then applies stochastic rejection sampling to retain only high-reward candidates (or preference pairs) for downstream optimization. We demonstrate how Ada-RS plugs into both preference pair (e.g. DPO) or grouped policy optimization strategies (e.g. DAPO). Using Qwen3-8B with LoRA on a synthetic tool call-oriented e-commerce benchmark, Ada-RS improves the accuracy-efficiency frontier over standard algorithms by reducing average output tokens by up to 80% and reducing thinking rate by up to 95% while maintaining or improving tool call accuracy. These results highlight that training-signal selection is a powerful lever for efficient reasoning in latency-sensitive deployments.