Reproducibility Companion Paper: Describing Subjective Experiment Consistency by $p$-Value P-P Plot

TL;DR

This paper reproduces and verifies the results of a previous study on subjective experiment consistency using $p$-value P-P plots, providing artifacts and software to facilitate reproducibility and further analysis.

Contribution

It offers reproducibility artifacts and software for subjective experiment analysis, validating prior results and supporting transparency in experimental research.

Findings

Results are fully reproducible using provided artifacts.

Software framework enables further subjective response analysis.

Reproducibility verified for all key figures and data.

Abstract

In this paper we reproduce experimental results presented in our earlier work titled "Describing Subjective Experiment Consistency by $p$-Value P-P Plot" that was presented in the course of the 28th ACM International Conference on Multimedia. The paper aims at verifying the soundness of our prior results and helping others understand our software framework. We present artifacts that help reproduce tables, figures and all the data derived from raw subjective responses that were included in our earlier work. Using the artifacts we show that our results are reproducible. We invite everyone to use our software framework for subjective responses analyses going beyond reproducibility efforts.