# In Situ Analysis of Electrodermal Activity from Students Learning from Large Language Models Versus Curated Texts

## TL;DR

This study compares learning from AI models versus curated texts by analyzing students' physiological responses and quiz results.

## Contribution

The study introduces in situ electrodermal activity analysis to assess learning effectiveness from Large Language Models.

## Key findings

- Learning with LLMs shows higher Skin Conductance Response linked to positive emotions.
- LLM-based learning correlates with higher quiz results compared to curated texts.

## Abstract

What are the main findings?
Generative AI and the use of Large Language Models have varying degrees of receptivity and impact in teaching and learning contexts.In this nascent field, studies have so far focused on medium-term usage patterns and their possible consequences.

Generative AI and the use of Large Language Models have varying degrees of receptivity and impact in teaching and learning contexts.

In this nascent field, studies have so far focused on medium-term usage patterns and their possible consequences.

What are the implications of the main findings?
This paper investigates the affective states of learners in situ as they are interacting with Large Language Models.The granularity of the in situ data differs from those obtained through post hoc means from prior work and offers insight into the efficacy (or lack thereof) of the use of Large Language Models in the arousal levels of learners.Implications for practice and/or policy: The use of low-cost, non-invasive wearables (which have been validated against industry-grade equipment) in the study suggests possibilities for future work to scale the investigation to other socio-cultural contexts of learning.

This paper investigates the affective states of learners in situ as they are interacting with Large Language Models.

The granularity of the in situ data differs from those obtained through post hoc means from prior work and offers insight into the efficacy (or lack thereof) of the use of Large Language Models in the arousal levels of learners.

Implications for practice and/or policy: The use of low-cost, non-invasive wearables (which have been validated against industry-grade equipment) in the study suggests possibilities for future work to scale the investigation to other socio-cultural contexts of learning.

Background: this paper reports an investigation into the cognitive and emotional states of adolescents while learning from an LLM. It seeks to address a relative dearth in empirical evidence which might otherwise facilitate informed decisions being made by curriculum designers, school leaders and policy makers regarding the use of Generative AI, amidst the wider discourse about the effectiveness of AI in teaching and learning. Methods: in this paper, we analyze electrodermal activity (EDA) in the context of students’ scholastic engagement using LLMs in comparison to curated texts. In our 27-min-long experiment, we recorded the EDA of participants learning from both learning methods, for 8 min each. A quiz was also conducted to assess the effectiveness of the learning method. We collected 23 samples of EDA from the experiment, and 42 samples of quiz results. Results: we have found that learning with an LLM results in greater Skin Conductance Response (p = 0.09404), which is linked to more positive emotional valence, and lower Skin Conductance Level (p = 0.09473), which is linked to lower cognitive load, compared to curated texts. We also discovered that learning with an LLM correlates to a higher quiz result (p = 0.02053). While this suggests that learning and absorbing information with an LLM could be more effective than curated texts, results from self-reported data indicate that there are few perceived differences between the effectiveness of LLM and curated texts. Conclusions: this exploratory and preliminary study revealed empirical insights between LLM usage and learning effectiveness in situ via physiological indicators, in contrast to prior work that has adopted post hoc frames over the medium- to long-term.