Finding Better Active Learners for Faster Literature Reviews

TL;DR

This paper introduces FASTREAD, an active learning approach that significantly accelerates literature reviews by reducing the number of papers needed to identify relevant studies, outperforming manual and existing automatic methods.

Contribution

The paper refactors and combines active learning techniques to create FASTREAD, a faster and more efficient method for systematic literature reviews.

Findings

FASTREAD finds 95% relevant studies after reviewing fewer papers.

FASTREAD reviews 20-50% fewer studies than other automatic methods.

FASTREAD matches the effectiveness of manual reviews in identifying relevant studies.

Abstract

Literature reviews can be time-consuming and tedious to complete. By cataloging and refactoring three state-of-the-art active learning techniques from evidence-based medicine and legal electronic discovery, this paper finds and implements FASTREAD, a faster technique for studying a large corpus of documents. This paper assesses FASTREAD using datasets generated from existing SE literature reviews (Hall, Wahono, Radjenovi\'c, Kitchenham et al.). Compared to manual methods, FASTREAD lets researchers find 95% relevant studies after reviewing an order of magnitude fewer papers. Compared to other state-of-the-art automatic methods, FASTREAD reviews 20-50% fewer studies while finding same number of relevant primary studies in a systematic literature review.