Evaluating the state-of-the-art in mapping research spaces: a Brazilian case study

TL;DR

This paper compares two methods for mapping research spaces using publication data, evaluating their predictive accuracy and sensitivity, and applying them to analyze Brazilian scientific dynamics.

Contribution

It provides a systematic comparison of frequentist and embedding-based models for research mapping using a large Brazilian dataset.

Findings

Embedding models outperform frequentist models in prediction accuracy.

Model sensitivity varies with publication count and field diversity.

Case study reveals insights into Brazil's scientific development.

Abstract

Scientific knowledge cannot be seen as a set of isolated fields, but as a highly connected network. Understanding how research areas are connected is of paramount importance for adequately allocating funding and human resources (e.g., assembling teams to tackle multidisciplinary problems). The relationship between disciplines can be drawn from data on the trajectory of individual scientists, as researchers often make contributions in a small set of interrelated areas. Two recent works propose methods for creating research maps from scientists' publication records: by using a frequentist approach to create a transition probability matrix; and by learning embeddings (vector representations). Surprisingly, these models were evaluated on different datasets and have never been compared in the literature. In this work, we compare both models in a systematic way, using a large dataset of publication records from Brazilian researchers. We evaluate these models' ability to predict whether a given entity (scientist, institution or region) will enter a new field w.r.t. the area under the ROC curve. Moreover, we analyze how sensitive each method is to the number of publications and the number of fields associated to one entity. Last, we conduct a case study to showcase how these models can be used to characterize science dynamics in the context of Brazil.