Logic Agent: Enhancing Validity with Logic Rule Invocation

TL;DR

This paper introduces the Logic Agent framework that enhances reasoning validity in large language models by dynamically invoking logic rules, leading to more coherent, interpretable, and precise reasoning across tasks.

Contribution

The paper presents a novel logic agent approach that transforms LLMs into logic-based reasoning systems with dynamic rule invocation, improving validity and interpretability.

Findings

Significant improvement in reasoning accuracy across tasks

Enhanced interpretability of reasoning processes

Effective scaling across different model sizes

Abstract

Chain-of-Thought (CoT) prompting has emerged as a pivotal technique for augmenting the inferential capabilities of language models during reasoning tasks. Despite its advancements, CoT often grapples with challenges in validating reasoning validity and ensuring informativeness. Addressing these limitations, this paper introduces the Logic Agent (LA), an agent-based framework aimed at enhancing the validity of reasoning processes in Large Language Models (LLMs) through strategic logic rule invocation. Unlike conventional approaches, LA transforms LLMs into logic agents that dynamically apply propositional logic rules, initiating the reasoning process by converting natural language inputs into structured logic forms. The logic agent leverages a comprehensive set of predefined functions to systematically navigate the reasoning process. This methodology not only promotes the structured and coherent generation of reasoning constructs but also significantly improves their interpretability and logical coherence. Through extensive experimentation, we demonstrate LA's capacity to scale effectively across various model sizes, markedly improving the precision of complex reasoning across diverse tasks.