Ranked-choice conjoint experiments

TL;DR

This paper introduces ranked-choice conjoint experiments, demonstrating their efficiency and providing an R package for implementation, with empirical evidence showing increased precision over forced-choice designs.

Contribution

It formalizes the integration of ranked outcomes into conjoint analysis, proving estimator equivalence, and offers practical tools and recommendations for researchers.

Findings

Ranked-choice conjoints produce similar but more precise AMCE estimates.

Adding profiles increases efficiency, reducing standard errors by up to 55%.

Recommended number of profiles is four for most applications.

Abstract

Forced-choice conjoint designs have become a staple method in the experimentalist's toolkit. However, the forced-choice outcome is neither always consistent with the types of choices individuals make in real political contexts, nor is it statistically efficient. In this paper, we formalize how ranked outcomes can be integrated into the conjoint framework. We provide a proof that rank-expanded estimators are equivalent to conventional AMCE, a theoretical account of how additional profiles increase the efficiency of conjoint designs, and design-based tests for the transitivity and independence of irrelevant alternatives assumptions that underpin the expansion. Across two pre-registered survey experiments--the first comparing forced-choice and ranked-choice designs across candidate and policy domains, and the second varying the number of ranked profiles--we find that ranked-choice conjoints yield substantively similar but more precise AMCE estimates, shrinking standard errors by 12-13% with one additional profile and up to 55% with six profiles per vignette. Based on efficiency--validity trade-offs, we recommend K = 4 profiles for most applications. We provide an accompanying open-source R package, cjrank, that implements rank expansion, AMCE estimation, efficiency diagnostics, and the assumption tests described in this paper.