LM-Scout: Analyzing the Security of Language Model Integration in Android Apps

TL;DR

This paper systematically studies insecure language model integration in Android apps, revealing widespread bypasses of restrictions, and introduces LM-Scout, an automated tool to detect and analyze these vulnerabilities at scale.

Contribution

It provides the first comprehensive analysis of LM security in Android apps, develops a taxonomy of restrictions, and presents LM-Scout for large-scale vulnerability detection.

Findings

127 out of 181 apps bypassed restrictions

LM-Scout detected vulnerabilities in 120 apps

Identified root causes and provided security recommendations

Abstract

Developers are increasingly integrating Language Models (LMs) into their mobile apps to provide features such as chat-based assistants. To prevent LM misuse, they impose various restrictions, including limits on the number of queries, input length, and allowed topics. However, if the LM integration is insecure, attackers can bypass these restrictions and gain unrestricted access to the LM, potentially harming developers' reputations and leading to significant financial losses. This paper presents the first systematic study of insecure usage of LMs by Android apps. We first manually analyze a preliminary dataset of apps to investigate LM integration methods, construct a taxonomy that categorizes the LM usage restrictions implemented by the apps, and determine how to bypass them. Alarmingly, we can bypass restrictions in 127 out of 181 apps. Then, we develop LM-Scout, a fully automated tool to detect on a large-scale vulnerable usage of LMs in 2,950 mobile apps. LM-Scout shows that, in many cases (i.e., 120 apps), it is possible to find and exploit such security issues automatically. Finally, we identify the root causes for the identified issues and offer recommendations for secure LM integration.