"Real Learner Data Matters" Exploring the Design of LLM-Powered Question Generation for Deaf and Hard of Hearing Learners

TL;DR

This paper investigates how Large Language Models can generate personalized, accessible quiz questions for Deaf and Hard of Hearing learners to improve their video-based education, emphasizing visual and emotional cues.

Contribution

It introduces a novel LLM-based question generation approach tailored for DHH learners, focusing on visual and emotional aspects to enhance learning experiences.

Findings

LLMs can generate relevant questions for DHH learners

Visual and emotion-based questions support diverse learning needs

Accessibility challenges remain for some DHH users

Abstract

Deaf and Hard of Hearing (DHH) learners face unique challenges in learning environments, often due to a lack of tailored educational materials that address their specific needs. This study explores the potential of Large Language Models (LLMs) to generate personalized quiz questions to enhance DHH students' video-based learning experiences. We developed a prototype leveraging LLMs to generate questions with emphasis on two unique strategies: Visual Questions, which identify video segments where visual information might be misrepresented, and Emotion Questions, which highlight moments where previous DHH learners experienced learning difficulty manifested in emotional responses. Through user studies with DHH undergraduates, we evaluated the effectiveness of these LLM-generated questions in supporting the learning experience. Our findings indicate that while LLMs offer significant potential for personalized learning, challenges remain in the interaction accessibility for the diverse DHH community. The study highlights the importance of considering language diversity and culture in LLM-based educational technology design.