An In-Depth Investigation of Data Collection in LLM App Ecosystems

TL;DR

This paper investigates data collection practices in LLM app ecosystems, revealing excessive data collection, policy violations, and poor transparency, and proposes an LLM-based framework for analyzing and auditing these practices.

Contribution

It introduces an LLM-based framework to analyze data collection practices and privacy policy consistency in LLM app ecosystems, exemplified through a case study of OpenAI's GPT ecosystem.

Findings

Actions collect excessive data across many categories

Several Actions violate privacy policies by collecting sensitive info

Most Actions do not clearly disclose their data collection practices

Abstract

LLM app (tool) ecosystems are rapidly evolving to support sophisticated use cases that often require extensive user data collection. Given that LLM apps are developed by third parties and anecdotal evidence indicating inconsistent enforcement of policies by LLM platforms, sharing user data with these apps presents significant privacy risks. In this paper, we aim to bring transparency in data practices of LLM app ecosystems. We examine OpenAI's GPT app ecosystem as a case study. We propose an LLM-based framework to analyze the natural language specifications of GPT Actions (custom tools) and assess their data collection practices. Our analysis reveals that Actions collect excessive data across 24 categories and 145 data types, with third-party Actions collecting 6.03% more data on average. We find that several Actions violate OpenAI's policies by collecting sensitive information, such as passwords, which is explicitly prohibited by OpenAI. Lastly, we develop an LLM-based privacy policy analysis framework to automatically check the consistency of data collection by Actions with disclosures in their privacy policies. Our measurements indicate that the disclosures for most of the collected data types are omitted, with only 5.8% of Actions clearly disclosing their data collection practices.