State of AI: An Empirical 100 Trillion Token Study with OpenRouter

TL;DR

This study analyzes over 100 trillion tokens from real-world LLM interactions to understand usage patterns, highlighting the popularity of open models, creative applications, and the importance of early user engagement.

Contribution

It provides the first large-scale empirical analysis of real-world LLM usage across diverse tasks and user cohorts, revealing complex engagement dynamics.

Findings

Open-weight models are widely adopted.

Creative roleplay and coding are prominent use cases.

Early users show longer engagement, termed the 'Glass Slipper' effect.

Abstract

The past year has marked a turning point in the evolution and real-world use of large language models (LLMs). With the release of the first widely adopted reasoning model, o1, on December 5th, 2024, the field shifted from single-pass pattern generation to multi-step deliberation inference, accelerating deployment, experimentation, and new classes of applications. As this shift unfolded at a rapid pace, our empirical understanding of how these models have actually been used in practice has lagged behind. In this work, we leverage the OpenRouter platform, which is an AI inference provider across a wide variety of LLMs, to analyze over 100 trillion tokens of real-world LLM interactions across tasks, geographies, and time. In our empirical study, we observe substantial adoption of open-weight models, the outsized popularity of creative roleplay (beyond just the productivity tasks many assume dominate) and coding assistance categories, plus the rise of agentic inference. Furthermore, our retention analysis identifies foundational cohorts: early users whose engagement persists far longer than later cohorts. We term this phenomenon the Cinderella "Glass Slipper" effect. These findings underscore that the way developers and end-users engage with LLMs "in the wild" is complex and multifaceted. We discuss implications for model builders, AI developers, and infrastructure providers, and outline how a data-driven understanding of usage can inform better design and deployment of LLM systems.