Web Question Answering with Neurosymbolic Program Synthesis

TL;DR

This paper introduces WebQA, a neurosymbolic program synthesis approach for web question answering that effectively generalizes from few labeled examples, outperforming existing methods across diverse domains.

Contribution

It presents a novel neurosymbolic DSL and an optimal, compositional synthesis algorithm with transductive learning for web information extraction.

Findings

WebQA outperforms state-of-the-art QA models.

The approach generalizes well across multiple domains.

The synthesis method efficiently finds optimal programs.

Abstract

In this paper, we propose a new technique based on program synthesis for extracting information from webpages. Given a natural language query and a few labeled webpages, our method synthesizes a program that can be used to extract similar types of information from other unlabeled webpages. To handle websites with diverse structure, our approach employs a neurosymbolic DSL that incorporates both neural NLP models as well as standard language constructs for tree navigation and string manipulation. We also propose an optimal synthesis algorithm that generates all DSL programs that achieve optimal F1 score on the training examples. Our synthesis technique is compositional, prunes the search space by exploiting a monotonicity property of the DSL, and uses transductive learning to select programs with good generalization power. We have implemented these ideas in a new tool called WebQA and evaluate it on 25 different tasks across multiple domains. Our experiments show that WebQA significantly outperforms existing tools such as state-of-the-art question answering models and wrapper induction systems.