RWebData: A High-Level Interface to the Programmable Web

TL;DR

RWebData is an R package that simplifies the process of collecting and preparing data from the programmable web for statistical analysis, making it accessible for social scientists without web technology experience.

Contribution

It introduces a high-level framework and a generic algorithm for parsing and mapping web data into flat tables, reducing technical barriers for R users.

Findings

Enables easy data collection from the web for R users

Provides a universal data mapping algorithm

Facilitates social science research with web data

Abstract

The rise of the programmable web offers new opportunities for the empirically driven social sciences. The access, compilation and preparation of data from the programmable web for statistical analysis can, however, involve substantial up-front costs for the practical researcher. The R-package RWebData provides a high-level framework that allows data to be easily collected from the programmable web in a format that can directly be used for statistical analysis in R (R Core Team 2013) without bothering about the data's initial format and nesting structure. It was developed specifically for users who have no experience with web technologies and merely use R as a statistical software. The core idea and methodological contribution of the package are the disentangling of parsing web data and mapping them with a generic algorithm (independent of the initial data structure) to a flat table-like representation. This paper provides an overview of the high-level functions for R-users, explains the basic architecture of the package, and illustrates the implemented data mapping algorithm.