Ranking Research Institutions Based On Related Academic Conferences

TL;DR

This paper presents a method for ranking research institutions within specific academic conferences using publication data and an aggregation approach, aiming to objectively identify influential institutions.

Contribution

It introduces an aggregation-based ranking method utilizing publication scores from multiple years to predict institutional influence in academic conferences.

Findings

Summing normalized scores across years closely approximated actual rankings.

The proposed method effectively predicted 2016 conference institution rankings.

Aggregation of multi-year data improved ranking accuracy.

Abstract

The detection of influential nodes in a social network is an active research area with many valuable applications including marketing and advertisement. As a new application in academia, KDD Cup 2016 shed light on the lack of an existing objective ranking for institutions within their respective research areas and proposed a solution for it. In this problem, the academic fields are defined as social networks whose nodes are the active institutions within the field, with the most influential nodes representing the highest contributors. The solution is able to provide a ranking of active institutions within their specific domains. The problem statement provided an annual scoring mechanism for institutions based on their publications and encouraged the use of any publicly available dataset such as the Microsoft Academic Graph (MAG). The contest was focused on research publications in selected conferences and asked for a prediction of the ranking for active institutions within those conferences in 2016. It should be noted that the results of the paper submissions and therefore the ground truths for KDD Cup were unknown at the time of the contest. Each team's final ranking list was evaluated by a metric called NDCG@20 after the results were released. This metric was used to indicate the distance between each team's proposed ranking and the actual one once it was known. After computing the scores of institutions for each year starting from 2011, we aggregated the rankings by summing the normalized scores across the years and using the final score set to provide the final ranking. Since the 2016 ground truths were unknown, we utilized the scores from 2011-2014 and used the 2015 publications as a test bed for evaluating our aggregation method. Based on the testing, summing the normalized scores got us closest to the actual 2015 rankings and using same heuristic for predicting the 2016 results.