Using Language Models to Detect Alarming Student Responses

TL;DR

This paper presents an AI-powered system utilizing fine-tuned language models to detect alarming student responses indicating potential threats or mental health risks, significantly improving accuracy over previous methods.

Contribution

The paper introduces a novel fine-tuned language model system integrated into assessment platforms for identifying concerning student responses, enhancing detection accuracy.

Findings

Substantial accuracy improvement over previous systems

Effective detection of threats and mental health risks

System integrated into assessment platforms

Abstract

This article details the advances made to a system that uses artificial intelligence to identify alarming student responses. This system is built into our assessment platform to assess whether a student's response indicates they are a threat to themselves or others. Such responses may include details concerning threats of violence, severe depression, suicide risks, and descriptions of abuse. Driven by advances in natural language processing, the latest model is a fine-tuned language model trained on a large corpus consisting of student responses and supplementary texts. We demonstrate that the use of a language model delivers a substantial improvement in accuracy over the previous iterations of this system.