Knowledge Questions from Knowledge Graphs

TL;DR

This paper presents an end-to-end method for automatically generating quiz questions from knowledge graphs, including difficulty estimation, with applications in education and knowledge assessment.

Contribution

It introduces a novel approach combining structured query generation, answer option selection, natural language verbalization, and difficulty prediction from large-scale data.

Findings

Effective question generation demonstrated on knowledge graphs.

Difficulty prediction model achieves accurate human difficulty estimation.

Approach supports multiple-choice question creation with high quality.

Abstract

We address the novel problem of automatically generating quiz-style knowledge questions from a knowledge graph such as DBpedia. Questions of this kind have ample applications, for instance, to educate users about or to evaluate their knowledge in a specific domain. To solve the problem, we propose an end-to-end approach. The approach first selects a named entity from the knowledge graph as an answer. It then generates a structured triple-pattern query, which yields the answer as its sole result. If a multiple-choice question is desired, the approach selects alternative answer options. Finally, our approach uses a template-based method to verbalize the structured query and yield a natural language question. A key challenge is estimating how difficult the generated question is to human users. To do this, we make use of historical data from the Jeopardy! quiz show and a semantically annotated Web-scale document collection, engineer suitable features, and train a logistic regression classifier to predict question difficulty. Experiments demonstrate the viability of our overall approach.