PRITES: An integrative framework for investigating and assessing web-scraped HTTP-response datasets for research applications

TL;DR

This paper introduces PRITES, an integrated framework for systematically documenting, assessing, and improving the quality of web-scraped datasets used in research, combining technical and statistical perspectives.

Contribution

The paper presents a novel, comprehensive framework (PRITES) that guides the entire process of web-scraped data collection and evaluation, bridging multiple disciplines.

Findings

Framework supports better documentation and quality assessment of web-scraped data.

Application of PRITES to retail price data demonstrates its practical utility.

Framework enhances transparency and reproducibility in web-scraped data research.

Abstract

The ability to programmatically retrieve vast quantities of data from online sources has given rise to increasing usage of web-scraped datasets for various purposes across government, industry and academia. Contemporaneously, there has also been growing discussion about the statistical qualities and limitations of collecting from online data sources and analysing web-scraped datasets. However, literature on web-scraping is distributed across computer science, statistical methodology and application domains, with distinct and occasionally conflicting definitions of web-scraping and conceptualisations of web-scraped data quality. This work synthesises technical and statistical concepts, best practices and insights across these relevant disciplines to inform documentation during web-scraping processes, and quality assessment of the resultant web-scraped datasets. We propose an integrated framework to cover multiple processes during the creation of web-scraped datasets including 'Plan', 'Retrieve', 'Investigate', 'Transform', 'Evaluate' and 'Summarise' (PRITES). The framework groups related quality factors which should be monitored during the collection of new web-scraped data, and/or investigated when assessing potential applications of existing web-scraped datasets. We connect each stage to existing discussions of technical and statistical challenges in collecting and analysing web-scraped data. We then apply the framework to describe related work by the co-authors to adapt web-scraped retail prices for alcoholic beverages collected by an industry data partner into analysis-ready datasets for public health policy research. The case study illustrates how the framework supports accurate and comprehensive scientific reporting of studies using web-scraped datasets.