Assessing the sustainable development of a national research ecosystem: A generative AI-based evaluation of empirical educational research in China (2004–2023)
Sen Wang, Yiming Wang
TL;DR
This study evaluates the sustainability of China's educational research system from 2004 to 2023 using generative AI, finding it generally robust but with room for improvement in data openness and methods.
Contribution
A novel generative AI-based framework for evaluating the sustainability of national educational research ecosystems.
Findings
China's educational research ecosystem has a sustainability index of 75.77, showing a maturing system.
Longitudinal trends reveal three stages: fluctuating development, rapid growth, and continuous improvement.
Lower scores for data transparency and accessibility highlight challenges in reproducibility and open science.
Abstract
A nation’s progress toward Sustainable Development Goal 4 (Quality Education) depends in part on the long-term health of its educational research system, yet systematic, longitudinal assessments of such research ecosystems remain scarce. This study applies a generative artificial intelligence–based framework to evaluate the sustainable development of China’s empirical educational research ecosystem from 2004 to 2023. We compiled a dataset of 2,145 empirical studies published in leading Chinese education journals and used GPT-4o to score each paper on 31 quality indicators covering research problem, theoretical framing, design, data collection, analysis, and reporting, using a 1–10 analytic rating scale. Based on the resulting score distributions, we constructed a fuzzy relation matrix and applied a fuzzy comprehensive evaluation method to derive annual and overall sustainability…
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17
Figure 18
Figure 19
Figure 20
Figure 21
Figure 22
Figure 23
Figure 24
Figure 25
Figure 26
Figure 27
Figure 28
Figure 29
Figure 30Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSustainability in Higher Education · E-Learning and COVID-19 · Educational Assessment and Improvement