# Data Analysis in Multimedia Quality Assessment: Revisiting the Statistical Tests

## TL;DR

This paper critically revisits statistical tests used in multimedia quality assessment, clarifying assumptions and providing practical insights supported by experiments, along with publicly available software for reproducibility.

## Contribution

It offers a theoretical review of parametric tests like t-test and ANOVA, highlighting common misconceptions and practical implications in multimedia quality analysis.

## Key findings

- Clarifies assumptions of normality and homogeneity of variance
- Provides practical guidelines for applying statistical tests
- Offers software tools for reproducible research

## Abstract

Assessment of multimedia quality relies heavily on subjective assessment, and is typically done by human subjects in the form of preferences or continuous ratings. Such data is crucial for analysis of different multimedia processing algorithms as well as validation of objective (computational) methods for the said purpose. To that end, statistical testing provides a theoretical framework towards drawing meaningful inferences, and making well grounded conclusions and recommendations. While parametric tests (such as t test, ANOVA, and error estimates like confidence intervals) are popular and widely used in the community, there appears to be a certain degree of confusion in the application of such tests. Specifically, the assumption of normality and homogeneity of variance is often not well understood. Therefore, the main goal of this paper is to revisit them from a theoretical perspective and in the process provide useful insights into their practical implications. Experimental results on both simulated and real data are presented to support the arguments made. A software implementing the said recommendations is also made publicly available, in order to achieve the goal of reproducible research.