Think Fast and Slow: Step-Level Cognitive Depth Adaptation for LLM Agents

TL;DR

This paper introduces CogRouter, a framework that enables large language model agents to dynamically adapt their cognitive depth at each step, improving efficiency and performance in decision-making tasks.

Contribution

It proposes a novel hierarchical cognitive level system grounded in ACT-R theory and a two-stage training method for step-level cognitive adaptation in LLM agents.

Findings

Achieves 82.3% success rate on ALFWorld and ScienceWorld

Outperforms GPT-4o, OpenAI-o3, and GRPO in success rate

Uses 62% fewer tokens than baseline models

Abstract

Large language models (LLMs) are increasingly deployed as autonomous agents for multi-turn decision-making tasks. However, current agents typically rely on fixed cognitive patterns: non-thinking models generate immediate responses, while thinking models engage in deep reasoning uniformly. This rigidity is inefficient for long-horizon tasks, where cognitive demands vary significantly from step to step, with some requiring strategic planning and others only routine execution. In this paper, we introduce CogRouter, a framework that trains agents to dynamically adapt cognitive depth at each step. Grounded in ACT-R theory, we design four hierarchical cognitive levels ranging from instinctive responses to strategic planning. Our two-stage training approach includes Cognition-aware Supervised Fine-tuning (CoSFT) to instill stable level-specific patterns, and Cognition-aware Policy Optimization (CoPO) for step-level credit assignment via confidence-aware advantage reweighting. The key insight is that appropriate cognitive depth should maximize the confidence of the resulting action. Experiments on ALFWorld and ScienceWorld demonstrate that CogRouter achieves state-of-the-art performance with superior efficiency. With Qwen2.5-7B, it reaches an 82.3% success rate, outperforming GPT-4o (+40.3%), OpenAI-o3 (+18.3%), and GRPO (+14.0%), while using 62% fewer tokens.