Analysis of the weighted kappa and its maximum with Markov moves

TL;DR

This paper uses Markov moves from Algebraic Statistics to analyze weighted kappa indices, explore their maximum values, and develop an algorithm to find optimal agreement configurations.

Contribution

It introduces a novel application of Markov moves to study weighted kappa indices and proposes a simulated annealing method to identify maximum agreement.

Findings

Markov moves effectively analyze weighted kappa indices.

The maximum kappa depends on the weighting scheme.

Simulated annealing successfully finds optimal agreement configurations.

Abstract

In this paper the notion of Markov move from Algebraic Statistics is used to analyze the weighted kappa indices in rater agreement problems. In particular, the problem of the maximum kappa and its dependence on the choice of the weighting schemes are discussed. The Markov moves are also used in a simulated annealing algorithm to actually find the configuration of maximum agreement.