Asking the Right Questions in Low Resource Template Extraction

TL;DR

This paper investigates how end users can effectively design questions for low-resource template extraction, proposing a novel prompt-based model that outperforms other prompt styles and does not require NLP expertise.

Contribution

It introduces a new prompt-based model for template extraction that leverages user-designed questions, demonstrating their effectiveness over other prompt formats and ease of use for non-experts.

Findings

Questions outperform other prompt styles in template extraction.

End users without NLP background can effectively design questions.

The proposed model improves data efficiency in low-resource settings.

Abstract

Information Extraction (IE) researchers are mapping tasks to Question Answering (QA) in order to leverage existing large QA resources, and thereby improve data efficiency. Especially in template extraction (TE), mapping an ontology to a set of questions can be more time-efficient than collecting labeled examples. We ask whether end users of TE systems can design these questions, and whether it is beneficial to involve an NLP practitioner in the process. We compare questions to other ways of phrasing natural language prompts for TE. We propose a novel model to perform TE with prompts, and find it benefits from questions over other styles of prompts, and that they do not require an NLP background to author.