Computational Approaches to Understanding Large Language Model Impact on Writing and Information Ecosystems

TL;DR

This paper explores how large language models are transforming writing and information ecosystems, examining biases, adoption patterns, and their potential to assist researchers, with implications for equity and governance.

Contribution

It introduces novel methods to measure LLM adoption, analyzes biases in AI detectors, and empirically evaluates LLMs' role in providing research feedback.

Findings

AI detectors introduce biases against non-dominant language writers

Widespread adoption of LLMs across multiple writing domains

LLMs can effectively support researchers with manuscript feedback

Abstract

Large language models (LLMs) have shown significant potential to change how we write, communicate, and create, leading to rapid adoption across society. This dissertation examines how individuals and institutions are adapting to and engaging with this emerging technology through three research directions. First, I demonstrate how the institutional adoption of AI detectors introduces systematic biases, particularly disadvantaging writers of non-dominant language varieties, highlighting critical equity concerns in AI governance. Second, I present novel population-level algorithmic approaches that measure the increasing adoption of LLMs across writing domains, revealing consistent patterns of AI-assisted content in academic peer reviews, scientific publications, consumer complaints, corporate communications, job postings, and international organization press releases. Finally, I investigate LLMs' capability to provide feedback on research manuscripts through a large-scale empirical analysis, offering insights into their potential to support researchers who face barriers in accessing timely manuscript feedback, particularly early-career researchers and those from under-resourced settings.