Question Answering via Integer Programming over Semi-Structured Knowledge

TL;DR

This paper introduces a structured inference system using Integer Linear Programming for answering science questions from semi-structured knowledge, significantly outperforming previous methods and enhancing robustness over statistical approaches.

Contribution

The paper presents a novel ILP-based inference system for science question answering that leverages semi-structured knowledge and outperforms prior structured reasoning methods.

Findings

Outperforms previous structured reasoning methods by 14%.

Improves ILP formulation accuracy by 17.7%.

Boosts overall performance by 10% when combined with unstructured methods.

Abstract

Answering science questions posed in natural language is an important AI challenge. Answering such questions often requires non-trivial inference and knowledge that goes beyond factoid retrieval. Yet, most systems for this task are based on relatively shallow Information Retrieval (IR) and statistical correlation techniques operating on large unstructured corpora. We propose a structured inference system for this task, formulated as an Integer Linear Program (ILP), that answers natural language questions using a semi-structured knowledge base derived from text, including questions requiring multi-step inference and a combination of multiple facts. On a dataset of real, unseen science questions, our system significantly outperforms (+14%) the best previous attempt at structured reasoning for this task, which used Markov Logic Networks (MLNs). It also improves upon a previous ILP formulation by 17.7%. When combined with unstructured inference methods, the ILP system significantly boosts overall performance (+10%). Finally, we show our approach is substantially more robust to a simple answer perturbation compared to statistical correlation methods.