Differentiating Between Human-Written and AI-Generated Texts Using Automatically Extracted Linguistic Features

TL;DR

This study compares linguistic features of human-written and AI-generated texts, revealing significant differences and highlighting the potential of automated tools for language assessment and AI training improvements.

Contribution

It systematically quantifies linguistic differences between human and AI texts using automated analysis, providing insights for enhancing AI language models.

Findings

Significant linguistic differences identified across phonological, morphological, syntactic, and lexical features.

AI-generated texts differ notably in the use of specific consonants, nouns, adjectives, and complex words.

Automated tools can effectively assist in language assessment and AI training processes.

Abstract

While extensive research has focused on ChatGPT in recent years, very few studies have systematically quantified and compared linguistic features between human-written and artificial intelligence (AI)-generated language. This exploratory study aims to investigate how various linguistic components are represented in both types of texts, assessing the ability of AI to emulate human writing. Using human-authored essays as a benchmark, we prompted ChatGPT to generate essays of equivalent length. These texts were analyzed using Open Brain AI, an online computational tool, to extract measures of phonological, morphological, syntactic, and lexical constituents. Despite AI-generated texts appearing to mimic human speech, the results revealed significant differences across multiple linguistic features such as specific types of consonants, nouns, adjectives, pronouns, adjectival/prepositional modifiers, and use of difficult words, among others. These findings underscore the importance of integrating automated tools for efficient language assessment, reducing time and effort in data analysis. Moreover, they emphasize the necessity for enhanced training methodologies to improve the engineering capacity of AI for producing more human-like text.