Machine Translation in the Wild: User Reaction to Xiaohongshu's Built-In Translation Feature

TL;DR

This study analyzes user reactions to Xiaohongshu's machine translation feature, revealing generally positive sentiments but highlighting concerns and unconventional testing behaviors that impact perceived effectiveness.

Contribution

It provides an empirical analysis of user perceptions and behaviors regarding a real-world MT feature on a social media platform, emphasizing the need for collaborative improvements.

Findings

Reactions were mostly positive but included concerns about accuracy and accessibility.

Users tested the MT with unconventional inputs like slang and encoded text.

Successful decoding led to positive responses, but conventional language testing was limited.

Abstract

This paper examines user reactions to the launch of the machine translation (MT) feature on Xiaohongshu, a Chinese social media and e-commerce platform, in January 2025. Drawing on a dataset of 6,723 comments collected from 11 official posts promoting the translation function, this paper combines sentiment analysis with thematic analysis to investigate how users perceived and experimented with the function. Results show that reactions were generally positive, although concerns regarding functionality, accessibility, and translation accuracy were also expressed. In addition, users actively tested the function with inputs that are atypical for everyday online communication, including stand-alone words and phrases, abbreviations, internet slang, and symbolic or encoded forms. Successful decoding of these texts elicited positive responses, while testing of more conventional language remained fairly limited. This could lead to uncritical acceptance of MT outputs by users, highlighting the importance of closer collaboration among computer scientists, translation scholars, and platform designers to improve MT performance and promote informed user engagement in real-world scenarios.