Nine-year-old children outperformed ChatGPT in emotion: Evidence from Chinese writing

TL;DR

This study compares Chinese writing abilities of nine-year-old children and ChatGPT, revealing children excel in emotion, fluency, and cohesion, while ChatGPT outperforms in accuracy and certain thematic complexities.

Contribution

It provides the first evidence that nine-year-old children express stronger emotions than ChatGPT in Chinese writing, highlighting differences in linguistic and emotional capabilities.

Findings

Children outperform ChatGPT in emotion, fluency, and cohesion.

ChatGPT has higher accuracy in Chinese writing.

Children excel in science-themed writing; ChatGPT in nature-themed writing.

Abstract

ChatGPT has been demonstrated to possess significant capabilities in generating intricate, human-like text, and recent studies have established that its performance in theory of mind tasks is comparable to that of a nine-year-old child. However, it remains uncertain whether ChatGPT surpasses nine-year-old children in Chinese writing proficiency. To explore this, our study juxtaposed the Chinese writing performance of ChatGPT and nine-year-old children on both narrative and scientific topics, aiming to uncover the relative strengths and weaknesses of ChatGPT in writing. The collected data were analyzed across five linguistic dimensions: fluency, accuracy, complexity, cohesion, and emotion. Each dimension underwent assessment through precise indices. The findings revealed that nine-year-old children excelled beyond ChatGPT in terms of fluency and cohesion within their writing. In contrast, ChatGPT manifested a superior performance in accuracy compared to the children. Concerning complexity, children exhibited superior skills in science-themed writing, while ChatGPT prevailed in nature-themed writing. Significantly, this research is pioneering in revealing that nine-year-old children convey stronger emotions than ChatGPT in their Chinese compositions.