Simple Yet Effective Methods for Large-Scale Scholarly Publication Ranking

TL;DR

This paper introduces a simple, transparent method for ranking scholarly publications, demonstrating competitive performance in a major challenge and discussing future directions for automatic evaluation techniques.

Contribution

The paper presents a novel, straightforward approach for large-scale scholarly publication ranking and analyzes its effectiveness and potential improvements.

Findings

Ranked among top performers in WSDM Cup 2016

Method is simple and transparent

Discusses future challenges in full-text evaluation

Abstract

With the growing amount of published research, automatic evaluation of scholarly publications is becoming an important task. In this paper we address this problem and present a simple and transparent approach for evaluating the importance of scholarly publications. Our method has been ranked among the top performers in the WSDM Cup 2016 Challenge. The first part of this paper describes our method. In the second part we present potential improvements to the method and analyse the evaluation setup which was provided during the challenge. Finally, we discuss future challenges in automatic evaluation of papers including the use of full-texts based evaluation methods.