Alloprof: a new French question-answer education dataset and its use in an information retrieval case study

TL;DR

This paper introduces a comprehensive French question-answering dataset from an educational website, and demonstrates its use in an information retrieval case study using BERT models, highlighting challenges and future directions.

Contribution

It provides a new large-scale French educational dataset with diverse content and a baseline retrieval approach, enabling research in French question-answering and multimodal comprehension.

Findings

The dataset contains 29,349 questions from students across various subjects.

Fine-tuned BERT models achieved acceptable retrieval performance on the dataset.

The dataset's complexity requires advanced algorithms for reliable educational applications.

Abstract

Teachers and students are increasingly relying on online learning resources to supplement the ones provided in school. This increase in the breadth and depth of available resources is a great thing for students, but only provided they are able to find answers to their queries. Question-answering and information retrieval systems have benefited from public datasets to train and evaluate their algorithms, but most of these datasets have been in English text written by and for adults. We introduce a new public French question-answering dataset collected from Alloprof, a Quebec-based primary and high-school help website, containing 29 349 questions and their explanations in a variety of school subjects from 10 368 students, with more than half of the explanations containing links to other questions or some of the 2 596 reference pages on the website. We also present a case study of this dataset in an information retrieval task. This dataset was collected on the Alloprof public forum, with all questions verified for their appropriateness and the explanations verified both for their appropriateness and their relevance to the question. To predict relevant documents, architectures using pre-trained BERT models were fine-tuned and evaluated. This dataset will allow researchers to develop question-answering, information retrieval and other algorithms specifically for the French speaking education context. Furthermore, the range of language proficiency, images, mathematical symbols and spelling mistakes will necessitate algorithms based on a multimodal comprehension. The case study we present as a baseline shows an approach that relies on recent techniques provides an acceptable performance level, but more work is necessary before it can reliably be used and trusted in a production setting.