Using Machine Learning and Natural Language Processing Techniques to Analyze and Support Moderation of Student Book Discussions

TL;DR

This paper presents a machine learning approach to automatically classify and moderate student book discussion messages, aiming to enhance online learning engagement and support teacher intervention.

Contribution

It introduces a novel feature stacking method and evaluates message classification techniques for discussion moderation in educational settings.

Findings

Feature stacking improves classification accuracy.

Machine learning methods are feasible for discussion moderation.

Bayesian t-test confirms significance of results.

Abstract

The increasing adoption of technology to augment or even replace traditional face-to-face learning has led to the development of a myriad of tools and platforms aimed at engaging the students and facilitating the teacher's ability to present new information. The IMapBook project aims at improving the literacy and reading comprehension skills of elementary school-aged children by presenting them with interactive e-books and letting them take part in moderated book discussions. This study aims to develop and illustrate a machine learning-based approach to message classification that could be used to automatically notify the discussion moderator of a possible need for an intervention and also to collect other useful information about the ongoing discussion. We aim to predict whether a message posted in the discussion is relevant to the discussed book, whether the message is a statement, a question, or an answer, and in which broad category it can be classified. We incrementally enrich our used feature subsets and compare them using standard classification algorithms as well as the novel Feature stacking method. We use standard classification performance metrics as well as the Bayesian correlated t-test to show that the use of described methods in discussion moderation is feasible. Moving forward, we seek to attain better performance by focusing on extracting more of the significant information found in the strong temporal interdependence of the messages.