Web-Browsing LLMs Can Access Social Media Profiles and Infer User Demographics

TL;DR

This paper demonstrates that web-browsing large language models can access social media profiles and accurately infer user demographics, revealing both potential applications and risks in social science and privacy.

Contribution

It is the first study to evaluate LLMs' ability to retrieve and analyze social media data for demographic inference using real-time web browsing capabilities.

Findings

LLMs can predict user demographics with reasonable accuracy.

Social media profile analysis reveals gender and political biases.

Risks include misuse in targeted advertising and information operations.

Abstract

Large language models (LLMs) have traditionally relied on static training data, limiting their knowledge to fixed snapshots. Recent advancements, however, have equipped LLMs with web browsing capabilities, enabling real time information retrieval and multi step reasoning over live web content. While prior studies have demonstrated LLMs ability to access and analyze websites, their capacity to directly retrieve and analyze social media data remains unexplored. Here, we evaluate whether web browsing LLMs can infer demographic attributes of social media users given only their usernames. Using a synthetic dataset of 48 X (Twitter) accounts and a survey dataset of 1,384 international participants, we show that these models can access social media content and predict user demographics with reasonable accuracy. Analysis of the synthetic dataset further reveals how LLMs parse and interpret social media profiles, which may introduce gender and political biases against accounts with minimal activity. While this capability holds promise for computational social science in the post API era, it also raises risks of misuse particularly in information operations and targeted advertising underscoring the need for safeguards. We recommend that LLM providers restrict this capability in public facing applications, while preserving controlled access for verified research purposes.