Reheat Nachos for Dinner? Evaluating AI Support for Cross-Cultural Communication of Neologisms

TL;DR

This study evaluates how AI tools assist non-native English speakers in understanding and using neologisms during cross-cultural communication, revealing strengths and limitations of current AI support methods.

Contribution

It provides empirical evidence on the effectiveness of different AI support types in improving communication of neologisms for non-native speakers.

Findings

AI Explanation support improves NS-rated communicative competence.

Contextual appropriateness judgments show no significant difference across AI support types.

NNS overestimate their own communicative competence compared to NS evaluations.

Abstract

Neologisms and emerging slang are central to daily conversation, yet challenging for non-native speakers (NNS) to interpret and use appropriately in cross-cultural communication with native speakers (NS). NNS increasingly make use of Artificial Intelligence (AI) tools to learn these words. We study the utility of such tools in mediating an informal communication scenario through a human-subjects study (N=234): NNS participants learn English neologisms with AI support, write messages using the learned word to an NS friend, and judge contextual appropriateness of the neologism in two provided writing samples. Using both NS evaluator-rated communicative competence of NNS-produced writing and NNS' contextual appropriateness judgments, we compare three AI-based support conditions: AI Definition, AI Rewrite into simpler English, AI Explanation of meaning and usage, and Non-AI Dictionary for comparison. We show that AI Explanation yields the largest gains over no support in NS-rated competence, while contextual appropriateness judgments show indifference across support. NNS participants' self-reported perceptions tend to overestimate NS ratings, revealing a mismatch between perceived and actual competence. We further observe a significant gap between NNS- and NS-produced writing, highlighting the limitations of current AI tools and informing design for future tools.