Machine Identification of High Impact Research through Text and Image Analysis

TL;DR

This paper introduces a system that combines visual and textual analysis to predict high-impact research papers, utilizing a large, diverse dataset across domains and time to improve generalization and early identification of influential work.

Contribution

It presents a novel dataset of PDFs and citation data spanning a decade across multiple domains, enabling more robust and generalizable impact prediction models.

Findings

The combined visual and text classifiers outperform single-modality models.

Models trained on one domain can partially predict impact in another domain.

The dataset facilitates research on impact prediction across time and disciplines.

Abstract

The volume of academic paper submissions and publications is growing at an ever increasing rate. While this flood of research promises progress in various fields, the sheer volume of output inherently increases the amount of noise. We present a system to automatically separate papers with a high from those with a low likelihood of gaining citations as a means to quickly find high impact, high quality research. Our system uses both a visual classifier, useful for surmising a document's overall appearance, and a text classifier, for making content-informed decisions. Current work in the field focuses on small datasets composed of papers from individual conferences. Attempts to use similar techniques on larger datasets generally only considers excerpts of the documents such as the abstract, potentially throwing away valuable data. We rectify these issues by providing a dataset composed of PDF documents and citation counts spanning a decade of output within two separate academic domains: computer science and medicine. This new dataset allows us to expand on current work in the field by generalizing across time and academic domain. Moreover, we explore inter-domain prediction models - evaluating a classifier's performance on a domain it was not trained on - to shed further insight on this important problem.