iotools: High-Performance I/O Tools for R

TL;DR

The iotools package enhances R's data processing capabilities with high-performance I/O functions, efficient parsing, and chunk-wise operations, demonstrated through practical use cases and extensive benchmarking against other packages.

Contribution

It introduces optimized I/O tools for R that reduce copying and intermediate representations, enabling efficient processing of large datasets and streaming data.

Findings

Significant performance improvements over core R functions.

Effective handling of large and streaming datasets.

Benchmark results favoring iotools over comparable packages.

Abstract

The iotools package provides a set of tools for Input/Output (I/O) intensive datasets processing in R (R Core Team, 2014). Efficent parsing methods are included which minimize copying and avoid the use of intermediate string representations whenever possible. Functions for applying chunk-wise operations allow for computing on streaming input as well as arbitrarily large files. We present a set of example use cases for iotools, as well as extensive benchmarks comparing comparable functions provided in both core-R as well as other contributed packages.