Enhancing Large Language Models through Structured Reasoning

TL;DR

This paper introduces a structured reasoning approach to improve large language models by converting unstructured data into explicit formats, fine-tuning with supervised methods, and applying novel algorithms to enhance reasoning and efficiency.

Contribution

It presents a new method combining explicit data annotation, supervised fine-tuning, and innovative algorithms to significantly enhance LLM reasoning capabilities.

Findings

Improved reasoning accuracy and robustness across scenarios

Enhanced compatibility with optimization techniques

Reduced computational complexity in reasoning tasks

Abstract

Recent Large Language Models (LLMs) have significantly advanced natural language processing and automated decision-making. However, these models still encounter difficulties when performing complex reasoning tasks involving logical deduction and systematic planning, primarily due to their reliance on implicit statistical relationships without structured knowledge representation.Inspired by cognitive science and neurosymbolic AI, we introduce a novel approach to enhance LLMs through explicit structured reasoning. First, we convert unstructured data into structured formats by explicitly annotating reasoning steps. We then employ this structured dataset to train LLMs through Supervised Fine-Tuning (SFT). Additionally, we enhance the structured reasoning capabilities of LLMs using Group Relative Policy Optimization (GRPO), incorporating two innovative algorithms--MAX-Flow and Longest Common Subsequence (LCS)--which notably improve reasoning effectiveness and reduce computational complexity. Experimental results from fine-tuning a DeepSeek-R1-Distill-Qwen-1.5B model demonstrate concise reasoning, robust performance across various scenarios, and improved compatibility with optimization techniques, validating the efficacy of structured reasoning integration in LLMs.