Language Detection Engine for Multilingual Texting on Mobile Devices
Sourabh Vasant Gothe, Sourav Ghosh, Sharmila Mani, Guggilla Bhanodai,, Ankur Agarwal, Chandramouli Sanchi

TL;DR
This paper introduces a fast, accurate, and lightweight language detection engine for multilingual mobile typing, significantly improving accuracy and speed over existing solutions, and effectively handling code-switching and language ambiguity.
Contribution
The paper presents a novel fusion of character N-gram and logistic regression models for real-time language detection on mobile devices, with parameter reduction for faster inference.
Findings
Achieves 94.5% accuracy for Indian languages in Latin script.
Attains 98% accuracy for European languages on code-switched data.
Outperforms fastText and ML-Kit in F1 score, with faster inference time.
Abstract
More than 2 billion mobile users worldwide type in multiple languages in the soft keyboard. On a monolingual keyboard, 38% of falsely auto-corrected words are valid in another language. This can be easily avoided by detecting the language of typed words and then validating it in its respective language. Language detection is a well-known problem in natural language processing. In this paper, we present a fast, light-weight and accurate Language Detection Engine (LDE) for multilingual typing that dynamically adapts to user intended language in real-time. We propose a novel approach where the fusion of character N-gram model and logistic regression based selector model is used to identify the language. Additionally, we present a unique method of reducing the inference time significantly by parameter reduction technique. We also discuss various optimizations fabricated across LDE to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsfastText · Logistic Regression
