Towards Neural Network-based Reasoning

TL;DR

Neural Reasoner is a neural network framework designed for reasoning over natural language, capable of handling complex logical relations and multiple facts, and it significantly outperforms previous systems on artificial reasoning tasks.

Contribution

The paper introduces Neural Reasoner with a novel interaction-pooling mechanism and deep architecture for flexible, end-to-end neural reasoning over natural language.

Findings

Outperforms existing neural reasoning systems on artificial tasks

Achieves over 98% accuracy on Path Finding task

Effectively models complex logical relations in reasoning

Abstract

We propose Neural Reasoner, a framework for neural network-based reasoning over natural language sentences. Given a question, Neural Reasoner can infer over multiple supporting facts and find an answer to the question in specific forms. Neural Reasoner has 1) a specific interaction-pooling mechanism, allowing it to examine multiple facts, and 2) a deep architecture, allowing it to model the complicated logical relations in reasoning tasks. Assuming no particular structure exists in the question and facts, Neural Reasoner is able to accommodate different types of reasoning and different forms of language expressions. Despite the model complexity, Neural Reasoner can still be trained effectively in an end-to-end manner. Our empirical studies show that Neural Reasoner can outperform existing neural reasoning systems with remarkable margins on two difficult artificial tasks (Positional Reasoning and Path Finding) proposed in [8]. For example, it improves the accuracy on Path Finding(10K) from 33.4% [6] to over 98%.