Asking Grok: AI-Assisted Sensemaking in Social Media Conversations

TL;DR

This study analyzes how users interact with the AI assistant Grok on social media platform X, revealing its role as an early sensemaking tool rather than a replacement for community fact-checking.

Contribution

It provides the first large-scale empirical analysis of user interactions with an LLM-powered AI assistant in real social media environments.

Findings

Grok is mainly used reactively for information verification.

Most users invoke Grok only once, indicating shallow adoption.

Interactions with Grok occur earlier than community fact-checking activities.

Abstract

LLM-powered AI assistants (e.g., Grok) are increasingly integrated into social media platforms, where they help explain content, provide context, and verify claims directly within conversation threads. While prior research has examined the accuracy of LLMs for fact-checking, little is known about how people interact with such systems in real-world social media environments. In this study, we empirically analyze user interactions with the AI assistant Grok on the social media platform X. Using a large-scale dataset consisting of 169,137 posts invoking Grok, we examine the types of requests directed at the AI assistant and the contexts in which it is used. We find that Grok is primarily invoked reactively to obtain or verify information. Although responses appear quickly, they typically only reach small audiences. Adoption is widespread but shallow, with 76.8% of users invoking Grok just once. We further examine how these interactions relate to Community Notes, X's community-based fact-checking system. While overlap between both systems is limited, it concentrates on verification-oriented and high-visibility content. Grok interactions typically occur earlier and do not predict subsequent correction activity. Together, these findings suggest that AI assistants function as an early complementary layer of sensemaking on social media rather than a replacement for crowd-based fact-checking systems.