Anticipating Technical Expertise and Capability Evolution in Research Communities using Dynamic Graph Transformers

TL;DR

This paper introduces a dynamic graph transformer model to predict evolving technical expertise and collaboration patterns in research communities, outperforming static models especially in emerging fields like AI and nuclear nonproliferation.

Contribution

The paper develops a novel dynamic graph transformer architecture that forecasts heterogeneous nodes and edges over time, advancing state-of-the-art in research community analysis.

Findings

DGT models predict collaboration and expertise with high accuracy.

Performance exceeds static baselines by 30-80%.

Models effectively predict collaborations involving new and early-career scientists.

Abstract

The ability to anticipate technical expertise and capability evolution trends globally is essential for national and global security, especially in safety-critical domains like nuclear nonproliferation (NN) and rapidly emerging fields like artificial intelligence (AI). In this work, we extend traditional statistical relational learning approaches (e.g., link prediction in collaboration networks) and formulate a problem of anticipating technical expertise and capability evolution using dynamic heterogeneous graph representations. We develop novel capabilities to forecast collaboration patterns, authorship behavior, and technical capability evolution at different granularities (e.g., scientist and institution levels) in two distinct research fields. We implement a dynamic graph transformer (DGT) neural architecture, which pushes the state-of-the-art graph neural network models by (a) forecasting heterogeneous (rather than homogeneous) nodes and edges, and (b) relying on both discrete -- and continuous -- time inputs. We demonstrate that our DGT models predict collaboration, partnership, and expertise patterns with 0.26, 0.73, and 0.53 mean reciprocal rank values for AI and 0.48, 0.93, and 0.22 for NN domains. DGT model performance exceeds the best-performing static graph baseline models by 30-80% across AI and NN domains. Our findings demonstrate that DGT models boost inductive task performance, when previously unseen nodes appear in the test data, for the domains with emerging collaboration patterns (e.g., AI). Specifically, models accurately predict which established scientists will collaborate with early career scientists and vice-versa in the AI domain.