LOGICPO: Efficient Translation of NL-based Logical Problems to FOL using LLMs and Preference Optimization

TL;DR

LOGICPO introduces a novel fine-tuning approach using preference optimization datasets and techniques like DPO and KTO to improve LLMs' ability to accurately translate natural language problems into logical forms, enhancing reasoning capabilities.

Contribution

The paper presents a new dataset and fine-tuning method that significantly improves LLMs' logical translation accuracy over existing models.

Findings

Phi-3.5 outperforms GPT-3.5-turbo by 10% in logical correctness.

Achieves 14% reduction in syntax errors.

Provides a new framework for better logical reasoning in LLMs.

Abstract

Logical reasoning is a key task for artificial intelligence due to it's role in major downstream tasks such as Question Answering, Summarization. Recent methods in improving the reasoning ability of LLMs fall short in correctly converting a natural language reasoning problem to an equivalent logical formulation, which hinders the framework's overall ability to reason. Towards this, we propose to use finetuning on a preference optimization dataset to learn to parse and represent a natural language problem as a whole to a consistent logical program by 1) introducing a new supervised and preference optimization dataset LogicPO, and 2) adopting popular techniques such as Direct Preference Optimization (DPO), Kahneman-Tversky optimization (KTO) to finetune open-source LLMs. Our best model with Phi-3.5 consistently outperforms GPT-3.5-turbo's (8-shot) by producing 10% more logically correct and with 14% less syntax errors. Through the framework and our improved evaluation metrics, we offer a promising direction in improving the logical reasoning of LLMs by better representing them in their logical formulations.