krippendorffsalpha: An R Package for Measuring Agreement Using Krippendorff's Alpha Coefficient

TL;DR

The paper introduces an R package that implements Krippendorff's Alpha for measuring agreement across various data types, supporting flexible analysis, user-defined functions, and bootstrap inference with parallel processing.

Contribution

It develops and illustrates an R package for Krippendorff's Alpha, enabling versatile agreement measurement with support for multiple data levels, missing data, and bootstrap inference.

Findings

Package supports nominal, ordinal, interval, ratio data

Allows user-defined distance functions

Enables bootstrap inference with parallel computation

Abstract

R package krippendorffsalpha provides tools for measuring agreement using Krippendorff's Alpha coefficient, a well-known nonparametric measure of agreement (also called inter-rater reliability and various other names). This article first develops Krippendorff's Alpha in a natural way, and situates Alpha among statistical procedures. Then the usage of package krippendorffsalpha is illustrated via analyses of two datasets, the latter of which was collected during an imaging study of hip cartilage. The package permits users to apply the Alpha methodology using built-in distance functions for the nominal, ordinal, interval, or ratio levels of measurement. User-defined distance functions are also supported. The fitting function can accommodate any number of units, any number of coders, and missingness. Bootstrap inference is supported, and the bootstrap computation can be carried out in parallel.