Logical Inference for Counting on Semi-structured Tables

TL;DR

This paper introduces a logical inference system that improves numerical reasoning in natural language inference tasks involving semi-structured tables, outperforming neural approaches in numerical understanding.

Contribution

The paper presents a novel logical inference approach using logical representations and model checking for numerical reasoning in semi-structured tables and texts.

Findings

System more robust in numerical inference than neural methods

Logical approach effectively handles numerical comparatives

Evaluation protocol demonstrates improved numerical understanding

Abstract

Recently, the Natural Language Inference (NLI) task has been studied for semi-structured tables that do not have a strict format. Although neural approaches have achieved high performance in various types of NLI, including NLI between semi-structured tables and texts, they still have difficulty in performing a numerical type of inference, such as counting. To handle a numerical type of inference, we propose a logical inference system for reasoning between semi-structured tables and texts. We use logical representations as meaning representations for tables and texts and use model checking to handle a numerical type of inference between texts and tables. To evaluate the extent to which our system can perform inference with numerical comparatives, we make an evaluation protocol that focuses on numerical understanding between semi-structured tables and texts in English. We show that our system can more robustly perform inference between tables and texts that requires numerical understanding compared with current neural approaches.