CAC-CoT: Connector-Aware Compact Chain-of-Thought for Efficient Reasoning Data Synthesis Across Dual-System Cognitive Tasks

TL;DR

CAC-CoT introduces a connector-aware, concise reasoning method for LLMs that improves efficiency and accuracy in dual-system tasks by restricting reasoning to fixed connector phrases, resulting in shorter traces and high performance.

Contribution

The paper proposes CAC-CoT, a simple yet effective approach that constrains reasoning to fixed connectors, enhancing reasoning efficiency and performance across System-1 and System-2 tasks.

Findings

Achieves 85% on GSM8K and S1-Bench, 40% on GPQA.

Reduces reasoning trace length to one-third of baseline.

Surpasses baseline performance by over 20%.

Abstract

Long chain-of-thought (CoT) prompting helps Large Language Models (LLMs) solve difficult problems, but very long traces often slow or even degrade performance on fast, intuitive "System-1" tasks. We introduce Connector-Aware Compact CoT (CAC-CoT) -- a method that deliberately restricts reasoning to a small, fixed set of connector phrases, steering the model toward concise and well -- structured explanations. Despite its simplicity, our synthetic method with general-purpose LLMs yields a high-quality training quality. CAC-CoT achieves approximately 85% on GSM8K and approximately 40% on GPQA (System-2) while also achieving approximately 85% on S1-Bench (System-1), surpassing the baseline by over 20%. Its reasoning traces average approximately 300 tokens(ART), about one-third the length of baseline traces, delivering higher efficiency without loss of accuracy.