LLM-Powered AI Tutors with Personas for d/Deaf and Hard-of-Hearing Online Learners

TL;DR

This study investigates how deaf and hard-of-hearing learners interact with LLM-powered AI tutors with personas, revealing their preferences, challenges, and the need for improved multimodal support and transparency.

Contribution

It introduces a user study exploring DHH learners' interactions with persona-based LLM tutors, highlighting accessibility needs and design implications for inclusive AI tutoring.

Findings

DHH learners ask culturally relevant questions about their community.

Participants desire more transparency about the AI tutors' community positions.

Current LLMs lack sufficient multimodal sign language support.

Abstract

Intelligent tutoring systems (ITS) using artificial intelligence (AI) technology have shown promise in supporting learners with diverse abilities. Large language models (LLMs) provide new opportunities to incorporate personas to AI-based tutors and support dynamic interactive dialogue. This paper explores how DHH learners interact with LLM-powered AI tutors with different experiences in DHH education as personas to identify their accessibility preferences. A user study with 16 DHH participants showed that they asked DHH-related questions based on background information and evaluated the AI tutors' cultural knowledge of the DHH communities in their responses. Participants suggested providing more transparency in each AI tutor's position within the DHH community. Participants also pointed out the lack of support in the multimodality of sign language in current LLMs. We discuss design implications to support the diverse needs in interaction between DHH users and the LLMs, such as offering supports in tuning language styles of LLMs.