An experimental approach: The graph of graphs

TL;DR

This study investigates optimal pairwise comparison patterns for decision-making, using empirical data from color preference experiments, and finds that empirical results align with previous simulations, providing practical tools and recommendations.

Contribution

It introduces an empirical analysis of comparison patterns, validating simulation results and offering a practical sequence and tools for decision problem assessments.

Findings

Empirical comparison patterns match simulated optimal patterns.

Most (close to) optimal comparison sequences are identical to previous simulations.

Provides practical recommendations and a Java application for decision-making processes.

Abstract

One of the essential issues in decision problems and preference modeling is the number of comparisons and their pattern to ask from the decision maker. We focus on the optimal patterns of pairwise comparisons and the sequence including the most (close to) optimal cases based on the results of a color selection experiment. In the test, six colors (red, green, blue, magenta, turquoise, yellow) were evaluated with pairwise comparisons as well as in a direct manner, on color-calibrated tablets in ISO standardized sensory test booths of a sensory laboratory. All the possible patterns of comparisons resulting in a connected representing graph were evaluated against the complete data based on 301 individual's pairwise comparison matrices (PCMs) using the logarithmic least squares weight calculation technique. It is shown that the empirical results, i.e., the empirical distributions of the elements of PCMs, are quite similar to the former simulated outcomes from the literature. The obtained empirically optimal patterns of comparisons were the best or the second best in the former simulations as well, while the sequence of comparisons that contains the most (close to) optimal patterns is exactly the same. In order to enhance the applicability of the results, besides the presentation of graph of graphs, and the representing graphs of the patterns that describe the proposed sequence of comparisons themselves, the recommendations are also detailed in a table format as well as in a Java application.