Diversifying Question Generation over Knowledge Base via External Natural Questions

TL;DR

This paper introduces a new diversity evaluation metric and a dual model framework for knowledge base question generation, significantly improving the diversity and relevance of generated questions and benefiting question answering tasks.

Contribution

It proposes a novel diversity metric and a dual model approach that leverages external natural questions to enhance question diversity in KBQG.

Findings

The new metric effectively measures diversity among top-k generated questions.

The dual model framework produces more diverse questions with high relevance.

Enhanced question diversity improves downstream question answering performance.

Abstract

Previous methods on knowledge base question generation (KBQG) primarily focus on enhancing the quality of a single generated question. Recognizing the remarkable paraphrasing ability of humans, we contend that diverse texts should convey the same semantics through varied expressions. The above insights make diversifying question generation an intriguing task, where the first challenge is evaluation metrics for diversity. Current metrics inadequately assess the above diversity since they calculate the ratio of unique n-grams in the generated question itself, which leans more towards measuring duplication rather than true diversity. Accordingly, we devise a new diversity evaluation metric, which measures the diversity among top-k generated questions for each instance while ensuring their relevance to the ground truth. Clearly, the second challenge is how to enhance diversifying question generation. To address this challenge, we introduce a dual model framework interwoven by two selection strategies to generate diverse questions leveraging external natural questions. The main idea of our dual framework is to extract more diverse expressions and integrate them into the generation model to enhance diversifying question generation. Extensive experiments on widely used benchmarks for KBQG demonstrate that our proposed approach generates highly diverse questions and improves the performance of question answering tasks.