Computing Information Agreement
Alberto Casagrande, Francesco Fabris, Rossano Girometti

TL;DR
This paper extends the Information Agreement measure, an information-theoretic tool for evaluating diagnostic agreement, to handle agreement matrices with zero values, overcoming previous limitations.
Contribution
It introduces a modification to the Information Agreement measure allowing zero values in agreement matrices, broadening its applicability.
Findings
Extended IA can now handle zero values in matrices
Maintains robustness compared to Cohen's Kappa
Improves evaluation of diagnostic systems
Abstract
Agreement measures are useful to both compare different evaluations of the same diagnostic outcomes and validate new rating systems or devices. Information Agreement (IA) is an information-theoretic-based agreement measure introduced to overcome all the limitations and alleged pitfalls of Cohen's Kappa. However, it is only able to deal with agreement matrices whose values are positive natural numbers. This work extends IA admitting also 0 as a possible value for the agreement matrix cells.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReliability and Agreement in Measurement · Rough Sets and Fuzzy Logic · Data Mining Algorithms and Applications
