CueBuddy: helping non-native English speakers navigate English-centric STEM education

TL;DR

CueBuddy is a real-time multilingual tool that helps non-native English-speaking STEM students understand technical jargon during lectures by providing lexical cues, addressing language barriers in resource-limited settings.

Contribution

This paper introduces CueBuddy, a novel system combining keyword spotting and glossary lookup to assist students with technical English in STEM education, especially in resource-constrained environments.

Findings

Effective real-time lexical cueing demonstrated in prototype

Addresses language barriers in STEM education for non-native speakers

Potential for scalable support in resource-limited settings

Abstract

Students across the world in STEM classes, especially in the Global South, fall behind their peers who are more fluent in English, despite being at par with them in terms of scientific prerequisites. While many of them are able to follow everyday English at ease, key terms in English stay challenging. In most cases, such students have had most of their course prerequisites in a lower resource language. Live speech translation to lower resource languages is a promising area of research, however, models for speech translation can be too expensive on a large scale and often struggle with technical content. In this paper, we describe CueBuddy, which aims to remediate these issues by providing real-time "lexical cues" through technical keyword spotting along real-time multilingual glossary lookup to help students stay up to speed with complex English jargon without disrupting their concentration on the lecture. We also describe the limitations and future extensions of our approach.