From 124 Million Tokens to 1,021 Neologisms: A Large-Scale Pipeline for Automatic Neologism Detection

TL;DR

This paper introduces a scalable, modular pipeline combining rule-based filtering and LLM classification to detect neologisms from a massive Reddit corpus, resulting in a verified set of 599 genuine lexical innovations.

Contribution

It presents a novel large-scale, modular pipeline for automatic neologism detection that integrates linguistic frameworks with machine learning, and provides a publicly available implementation.

Findings

Extracted 124.6 million tokens from Reddit posts and identified 1,021 neologism candidates.

Manual verification confirmed 599 (58.7%) candidates as genuine neologisms.

Multiple LLMs showed significant disagreement, highlighting challenges in large-scale neologism detection.

Abstract

We present a scalable, modular pipeline for automatic neologism detection that combines rule-based filtering with LLM classification. The pipeline is grounded in two complementary word-formation frameworks, grammatical and extra-grammatical morphology, which jointly define the scope of what counts as a neologism and inform a four-class classification scheme (neologism, entity, foreign, none). While designed to be modular and transferable at the architectural level, the pipeline is instantiated on 527 million English-language Reddit posts spanning 2005-2024. From this corpus, we extract 124.6 million unique tokens and reduce them by over 99.99% to yield 1,021 neologism candidates, a set small enough for manual expert verification. Multiple LLMs independently classify each candidate via majority vote, with a final verification step, revealing substantial cross-model disagreement and highlighting the challenge of operationalizing neologism detection at scale. Manual annotation of all 1,021 candidates confirms that 599 (58.7%) are genuine lexical innovations. The pipeline code, vocabulary compilation scripts, and the annotated candidate list are available at https://github.com/DiegoRossini/neologism-pipeline.