Latent Attention For If-Then Program Synthesis

TL;DR

This paper introduces Latent Attention, a neural network architecture that improves the translation of natural language descriptions into If-Then programs, achieving a 28.57% error reduction and exploring one-shot learning scenarios.

Contribution

The paper presents a novel Latent Attention mechanism for end-to-end neural program synthesis from natural language, specifically targeting If-Then programs, and demonstrates its effectiveness.

Findings

Error rate reduced by 28.57% compared to prior methods.

Proposed one-shot learning scenario improves model performance.

Training procedure variations close the gap to full-data models.

Abstract

Automatic translation from natural language descriptions into programs is a longstanding challenging problem. In this work, we consider a simple yet important sub-problem: translation from textual descriptions to If-Then programs. We devise a novel neural network architecture for this task which we train end-to-end. Specifically, we introduce Latent Attention, which computes multiplicative weights for the words in the description in a two-stage process with the goal of better leveraging the natural language structures that indicate the relevant parts for predicting program elements. Our architecture reduces the error rate by 28.57% compared to prior art. We also propose a one-shot learning scenario of If-Then program synthesis and simulate it with our existing dataset. We demonstrate a variation on the training procedure for this scenario that outperforms the original procedure, significantly closing the gap to the model trained with all data.