FSLI: An Interpretable Formal Semantic System for One-Dimensional Ordering Inference

TL;DR

FSLI is an interpretable logical deduction system that transforms natural language into first-order logic, achieving high accuracy on logical reasoning tasks and emphasizing symbolic over neural approaches.

Contribution

The paper introduces FSLI, a formal semantic system that uses lambda calculus and constraint logic programming for natural language logical deduction, with superior accuracy on benchmark tasks.

Findings

100% accuracy on BIG-bench logical deduction task

88% accuracy on simplified AR-LSAT subset

Outperforms neural language model baseline

Abstract

We develop a system for solving logical deduction one-dimensional ordering problems by transforming natural language premises and candidate statements into first-order logic. Building on Heim and Kratzer's syntax-based compositional semantic rules which utilizes lambda calculus, we develop a semantic parsing algorithm with abstract types, templated rules, and a dynamic component for interpreting entities within a context constructed from the input. The resulting logical forms are executed via constraint logic programming to determine which candidate statements can be logically deduced from the premises. The symbolic system, the Formal Semantic Logic Inferer (FSLI), provides a formally grounded, linguistically driven system for natural language logical deduction. We evaluate it on both synthetic and derived logical deduction problems. FSLI achieves 100% accuracy on BIG-bench's logical deduction task and 88% on a syntactically simplified subset of AR-LSAT outperforming an LLM baseline, o1-preview. While current research in natural language reasoning emphasizes neural language models, FSLI highlights the potential of principled, interpretable systems for symbolic logical deduction in NLP.