Shallow Synthesis of Knowledge in GPT-Generated Texts: A Case Study in Automatic Related Work Composition

TL;DR

This study evaluates GPT-4's ability to generate related work sections in academic papers, finding it can produce coarse citation groupings but lacks detailed synthesis, emphasizing the need for human oversight in AI-assisted scholarly writing.

Contribution

The paper introduces a method to analyze AI-generated scholarly texts, specifically assessing GPT-4's capabilities in synthesizing related work sections through citation graph analysis.

Findings

GPT-4 can produce coarse citation groupings for brainstorming.

GPT-4 struggles with detailed synthesis of related works.

Human intervention improves synthesis quality.

Abstract

Numerous AI-assisted scholarly applications have been developed to aid different stages of the research process. We present an analysis of AI-assisted scholarly writing generated with ScholaCite, a tool we built that is designed for organizing literature and composing Related Work sections for academic papers. Our evaluation method focuses on the analysis of citation graphs to assess the structural complexity and inter-connectedness of citations in texts and involves a three-way comparison between (1) original human-written texts, (2) purely GPT-generated texts, and (3) human-AI collaborative texts. We find that GPT-4 can generate reasonable coarse-grained citation groupings to support human users in brainstorming, but fails to perform detailed synthesis of related works without human intervention. We suggest that future writing assistant tools should not be used to draft text independently of the human author.