Learning a Natural Language Interface with Neural Programmer

TL;DR

This paper introduces a weakly supervised neural network model called Neural Programmer that learns to generate programs for natural language questions on tables, achieving competitive accuracy without domain-specific rules.

Contribution

It presents the first end-to-end neural model for program induction on real-world data using weak supervision, eliminating the need for handcrafted grammars or annotations.

Findings

Achieves 34.2% accuracy with 10,000 examples

Ensemble of 15 models reaches 37.7% accuracy

Competitive with state-of-the-art semantic parsers

Abstract

Learning a natural language interface for database tables is a challenging task that involves deep language understanding and multi-step reasoning. The task is often approached by mapping natural language queries to logical forms or programs that provide the desired response when executed on the database. To our knowledge, this paper presents the first weakly supervised, end-to-end neural network model to induce such programs on a real-world dataset. We enhance the objective function of Neural Programmer, a neural network with built-in discrete operations, and apply it on WikiTableQuestions, a natural language question-answering dataset. The model is trained end-to-end with weak supervision of question-answer pairs, and does not require domain-specific grammars, rules, or annotations that are key elements in previous approaches to program induction. The main experimental result in this paper is that a single Neural Programmer model achieves 34.2% accuracy using only 10,000 examples with weak supervision. An ensemble of 15 models, with a trivial combination technique, achieves 37.7% accuracy, which is competitive to the current state-of-the-art accuracy of 37.1% obtained by a traditional natural language semantic parser.