What makes us curious? analysis of a corpus of open-domain questions

TL;DR

This paper analyzes a large corpus of open-domain questions to understand curiosity and develops QBERT, a generalist AI model for question analysis tasks like equivalence detection, topic classification, and answering.

Contribution

It introduces QBERT, a versatile AI model trained on diverse datasets for comprehensive question analysis in open-domain contexts.

Findings

Successfully extracted insights about curiosity from 10,000 questions

Developed QBERT, capable of detecting question equivalence, topics, and providing answers

Demonstrated the model's effectiveness across various open-domain question types

Abstract

Every day people ask short questions through smart devices or online forums to seek answers to all kinds of queries. With the increasing number of questions collected it becomes difficult to provide answers to each of them, which is one of the reasons behind the growing interest in automated question answering. Some questions are similar to existing ones that have already been answered, while others could be answered by an external knowledge source such as Wikipedia. An important question is what can be revealed by analysing a large set of questions. In 2017, "We the Curious" science centre in Bristol started a project to capture the curiosity of Bristolians: the project collected more than 10,000 questions on various topics. As no rules were given during collection, the questions are truly open-domain, and ranged across a variety of topics. One important aim for the science centre was to understand what concerns its visitors had beyond science, particularly on societal and cultural issues. We addressed this question by developing an Artificial Intelligence tool that can be used to perform various processing tasks: detection of equivalence between questions; detection of topic and type; and answering of the question. As we focused on the creation of a "generalist" tool, we trained it with labelled data from different datasets. We called the resulting model QBERT. This paper describes what information we extracted from the automated analysis of the WTC corpus of open-domain questions.