Comment on: Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Tasks

TL;DR

This paper comments on a recent study about AI's impact on human cognitive performance, highlighting methodological concerns and emphasizing the need for more rigorous and transparent research practices.

Contribution

It provides constructive critique of the original study, aiming to improve research quality and reproducibility in AI-human performance investigations.

Findings

Highlights limitations in sample size and reproducibility

Raises methodological concerns in EEG analysis

Calls for greater transparency in research procedures

Abstract

Recently published work titled Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Task by Kosmyna et al. (2025) has sparked a vivid debate on the topic of artificial intelligence (AI) and human performance. We sincerely congratulate Kosmyna et al. for initiating such important research, collecting a valuable dataset, and establishing highly automated pipelines for Natural Language Processing (NLP) analyses and scoring. We aim to provide constructive comments that may improve the manuscript's readiness for peer-reviewed publication, as some results by Kosmyna et al. (2025) could be interpreted more conservatively. Our primary concerns focus on: (i) study design considerations, including the limited sample size; (ii) the reproducibility of the analyses; (iii) methodological issues related to the EEG analysis; (iv) inconsistencies in the reporting of results; and (v) limited transparency in several aspects of the study's procedures and findings.