Internal and External Impacts of Natural Language Processing Papers

TL;DR

This study analyzes the influence of NLP research from 1979 to 2024 across academic citations and external sources, highlighting the varying impacts of different topics and the societal relevance of recent concerns.

Contribution

It provides a comprehensive analysis of how NLP research impacts both academic and external domains, emphasizing the influence of language modeling and societal issues.

Findings

Language modeling has the broadest influence internally and externally.

Linguistic foundations have comparatively lower impacts.

Topics like ethics and bias attract more policy attention than academic citations.

Abstract

We investigate the impacts of NLP research published in top-tier conferences (i.e., ACL, EMNLP, and NAACL) from 1979 to 2024. By analyzing citations from research articles and external sources such as patents, media, and policy documents, we examine how different NLP topics are consumed both within the academic community and by the broader public. Our findings reveal that language modeling has the widest internal and external influence, while linguistic foundations have lower impacts. We also observe that internal and external impacts generally align, but topics like ethics, bias, and fairness show significant attention in policy documents with much fewer academic citations. Additionally, external domains exhibit distinct preferences, with patents focusing on practical NLP applications and media and policy documents engaging more with the societal implications of NLP models.