Exploring Self-supervised Logic-enhanced Training for Large Language Models

TL;DR

This paper introduces LogicLLM, a self-supervised logic-enhanced training method for large language models, improving their logical reasoning capabilities beyond traditional supervised fine-tuning.

Contribution

It proposes a novel self-supervised post-training approach using a logic-oriented proxy task, enhancing LLMs' reasoning abilities without supervised data.

Findings

LogicLLM outperforms baseline models on logical reasoning benchmarks.

Self-supervised logic training improves LLM reasoning capabilities.

Extensive ablations identify key factors in logic proxy task design.

Abstract

Existing efforts to improve logical reasoning ability of language models have predominantly relied on supervised fine-tuning, hindering generalization to new domains and/or tasks. The development of Large Langauge Models (LLMs) has demonstrated the capacity of compressing abundant knowledge into a single proxy, enabling them to tackle multiple tasks effectively. Our preliminary experiments, nevertheless, show that LLMs do not show capability on logical reasoning. The performance of LLMs on logical reasoning benchmarks is far behind the existing state-of-the-art baselines. In this paper, we make the first attempt to investigate the feasibility of incorporating logical knowledge through self-supervised post-training, and activating it via in-context learning, which we termed as LogicLLM. Specifically, we devise an auto-regressive objective variant of MERIt and integrate it with two LLM series, i.e., FLAN-T5 and LLaMA, with parameter size ranging from 3 billion to 13 billion. The results on two challenging logical reasoning benchmarks demonstrate the effectiveness of LogicLLM. Besides, we conduct extensive ablation studies to analyze the key factors in designing logic-oriented proxy tasks.