CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning

TL;DR

CoMAT introduces a two-stage reasoning approach that converts natural language problems into symbolic form and executes reasoning within a single LLM, significantly improving mathematical reasoning accuracy and transparency.

Contribution

It proposes CoMAT, a novel method that enhances mathematical reasoning by combining symbolic conversion and reasoning execution within one LLM, without external tools.

Findings

Outperforms traditional CoT on six of seven benchmarks.

Achieves 4.48% improvement on MMLU-Redux (MATH).

Achieves 4.58% improvement on GaoKao MCQ.

Abstract

Mathematical reasoning remains a significant challenge for large language models (LLMs), despite progress in prompting techniques such as Chain-of-Thought (CoT). We present **Chain of Mathematically Annotated Thought (CoMAT)**, which enhances reasoning through two stages: *Symbolic Conversion* (converting natural language queries into symbolic form) and *Reasoning Execution* (deriving answers from symbolic representations). CoMAT operates entirely with a single LLM and without external solvers. Across four LLMs, CoMAT outperforms traditional CoT on six out of seven benchmarks, achieving gains of 4.48% on MMLU-Redux (MATH) and 4.58% on GaoKao MCQ. In addition to improved performance, CoMAT ensures faithfulness and verifiability, offering a transparent reasoning process for complex mathematical tasks