The Empowerment of Science of Science by Large Language Models: New Tools and Methods

TL;DR

This paper reviews how large language models (LLMs) enhance the Science of Science by introducing new tools, methods, and applications, including scientometric analysis, knowledge graphs, and AI-based scientific evaluation.

Contribution

It provides a comprehensive overview of LLM technologies and explores their potential to transform scientometric research and scientific evaluation methods.

Findings

Introduction of new research fronts detection methods

Development of knowledge graph building techniques

Proposal of AI agent-based models for scientific evaluation

Abstract

Large language models (LLMs) have exhibited exceptional capabilities in natural language understanding and generation, image recognition, and multimodal tasks, charting a course towards AGI and emerging as a central issue in the global technological race. This manuscript conducts a comprehensive review of the core technologies that support LLMs from a user standpoint, including prompt engineering, knowledge-enhanced retrieval augmented generation, fine tuning, pretraining, and tool learning. Additionally, it traces the historical development of Science of Science (SciSci) and presents a forward looking perspective on the potential applications of LLMs within the scientometric domain. Furthermore, it discusses the prospect of an AI agent based model for scientific evaluation, and presents new research fronts detection and knowledge graph building methods with LLMs.