Building benchmarking frameworks for supporting replicability and reproducibility: spatial and textual analysis as an example

TL;DR

This paper advocates for developing benchmarking frameworks in GIScience to enhance replicability and reproducibility, using spatial and textual analysis, exemplified through geoparsing, to enable effective comparison of methods and tools.

Contribution

It introduces the concept of benchmarking frameworks in GIScience to support R&R, demonstrated through the example of geoparsing in spatial and textual analysis.

Findings

Benchmarking frameworks facilitate research replication.

They enable effective comparison of methods and tools.

Geoparsing serves as a practical example.

Abstract

Replicability and reproducibility (R&R) are critical for the long-term prosperity of a scientific discipline. In GIScience, researchers have discussed R&R related to different research topics and problems, such as local spatial statistics, digital earth, and metadata (Fotheringham, 2009; Goodchild, 2012; Anselin et al., 2014). This position paper proposes to further support R&R by building benchmarking frameworks in order to facilitate the replication of previous research for effective and effcient comparisons of methods and software tools developed for addressing the same or similar problems. Particularly, this paper will use geoparsing, an important research problem in spatial and textual analysis, as an example to explain the values of such benchmarking frameworks.