What Does AI Do for Cultural Interpretation? A Randomized Experiment on Close Reading Poems with Exposure to AI Interpretation

TL;DR

This study investigates how AI assistance influences human close reading of poems, finding that a single AI interpretation enhances both interpretative accuracy and enjoyment, while multiple AI inputs improve performance but may reduce pleasure.

Contribution

It provides empirical evidence on the effects of AI assistance in cultural interpretation, highlighting the benefits and trade-offs of different levels of AI support.

Findings

Single AI interpretation improves performance and pleasure.

Multiple AI interpretations enhance performance but may decrease pleasure.

Heavy reliance on AI correlates with better performance but lower enjoyment.

Abstract

AI demonstrates unprecedented reasoning capabilities, but its increasing integration into human reasoning via automated reading and summarization has provoked debate about its use for cultural interpretation. Close reading -- the practice of understanding, analyzing, and critiquing cultural texts for pleasure -- is a skill at the core of such interpretation, traditionally being seen as exclusive to humans. To test AI's impact on close reading, both in terms of interpretative performance and pleasure, we conducted a preregistered randomized experiment (n=400) investigating the impact of AI assistance by presenting single or multiple AI interpretations, on close reading poems, compared to no AI assistance. We found that single AI interpretation boosted both performance and pleasure, while multiple AI interpretations only improved performance. Further exploration revealed a trade-off: participants who heavily relied on AI showed better performance on the task but lower pleasure. Our results contribute to discussion on whether and how to calibrate AI assistance for cultural interpretation: "less is more."