A Reduced Basis Decomposition Approach to Efficient Data Collection in Pairwise Comparison Studies

TL;DR

This paper introduces a scalable reduced basis decomposition method for designing efficient pairwise comparison studies, significantly reducing computational costs while maintaining high approximation quality, enabling real-time experimental design updates.

Contribution

The paper presents a novel reduced basis decomposition approach that bypasses spectral decomposition, enabling fast and scalable experimental design in large pairwise comparison studies.

Findings

Achieves 100x speedup in design computation for large studies

Maintains negligible approximation error with reduced basis method

Enables real-time design updates in practical applications

Abstract

Comparative judgement studies elicit quality assessments through pairwise comparisons, typically analysed using the Bradley-Terry model. A challenge in these studies is experimental design, specifically, determining the optimal pairs to compare to maximize statistical efficiency. Constructing static experimental designs for these studies requires spectral decomposition of a covariance matrix over pairs of pairs, which becomes computationally infeasible for studies with more than approximately 150 objects. We propose a scalable method based on reduced basis decomposition that bypasses explicit construction of this matrix, achieving computational savings of two to three orders of magnitude. We establish eigenvalue bounds guaranteeing approximation quality and characterise the rank structure of the design matrix. Simulations demonstrate speedup factors exceeding 100 for studies with 64 or more objects, with negligible approximation error. We apply the method to construct designs for a 452-region spatial study in under 7 minutes and enable real-time design updates for classroom peer assessment, reducing computation time from 15 minutes to 15 seconds.