Decoding Student Minds: Leveraging Conversational Agents for Psychological and Learning Analysis

TL;DR

This paper introduces a psychologically-aware conversational agent that uses multimodal data and advanced NLP techniques to assess and improve students' cognitive and emotional states in real time, enhancing learning and well-being.

Contribution

It presents a novel multimodal, real-time student state classification system combining LLMs, KG-BERT, and LSTM with attention, surpassing prior chatbots limited to single-function support.

Findings

Improved student motivation and reduced stress in pilot study

Moderate academic gains observed with the system

Effective real-time classification of engagement, stress, and understanding

Abstract

This paper presents a psychologically-aware conversational agent designed to enhance both learning performance and emotional well-being in educational settings. The system combines Large Language Models (LLMs), a knowledge graph-enhanced BERT (KG-BERT), and a bidirectional Long Short-Term Memory (LSTM) with attention to classify students' cognitive and affective states in real time. Unlike prior chatbots limited to either tutoring or affective support, our approach leverages multimodal data-including textual semantics, prosodic speech features, and temporal behavioral trends-to infer engagement, stress, and conceptual understanding. A pilot study with university students demonstrated improved motivation, reduced stress, and moderate academic gains compared to baseline methods. These results underline the promise of integrating semantic reasoning, multimodal fusion, and temporal modeling to support adaptive, student-centered educational interventions.