Design Opportunities for Explainable AI Paraphrasing Tools: A User Study with Non-native English Speakers

TL;DR

This study explores how non-native English speakers interact with explainable AI paraphrasing tools, revealing diverse preferences and workflows that inform better design for supporting language learners.

Contribution

It introduces ParaScope, an AI paraphrasing assistant with integrated information aids, and provides empirical insights into user interactions and preferences among NNESs.

Findings

Back-translation was the most used aid but not decisive for acceptance.

User workflows vary from global to detailed information based on proficiency.

Multiple aids are combined to support informed decision-making.

Abstract

We investigate how non-native English speakers (NNESs) interact with diverse information aids to assess and select AI-generated paraphrases. We develop ParaScope, an AI paraphrasing assistant that integrates diverse information aids, such as back-translation, explanations, and usage examples, and logs user interaction data. Our in-lab study with 22 NNESs reveals that user preferences for information aids vary by language proficiency, with workflows progressing from global to more detailed information. While back-translation was the most frequently used aid, it was not a decisive factor in suggestion acceptance; users combined multiple information aids to make informed decisions. Our findings demonstrate the potential of explainable AI paraphrasing tools to enhance NNESs' confidence, autonomy, and writing efficiency, while also emphasizing the importance of thoughtful design to prevent information overload. Based on these findings, we offer design implications for explainable AI paraphrasing tools that support NNESs in making informed decisions when using AI writing systems.