Dynamic Question Ordering in Online Surveys

TL;DR

This paper introduces a framework for personalized dynamic question ordering in online surveys, aiming to enhance engagement, response rates, and prediction accuracy by adaptively selecting questions based on previous answers.

Contribution

It proposes a novel DQO framework that personalizes question sequences to improve survey completion and prediction quality, extending beyond rule-based methods.

Findings

Framework effectively increases survey completion rates.

Personalized question ordering improves prediction accuracy.

Application to energy estimates demonstrates practical benefits.

Abstract

Online surveys have the potential to support adaptive questions, where later questions depend on earlier responses. Past work has taken a rule-based approach, uniformly across all respondents. We envision a richer interpretation of adaptive questions, which we call dynamic question ordering (DQO), where question order is personalized. Such an approach could increase engagement, and therefore response rate, as well as imputation quality. We present a DQO framework to improve survey completion and imputation. In the general survey-taking setting, we want to maximize survey completion, and so we focus on ordering questions to engage the respondent and collect hopefully all information, or at least the information that most characterizes the respondent, for accurate imputations. In another scenario, our goal is to provide a personalized prediction. Since it is possible to give reasonable predictions with only a subset of questions, we are not concerned with motivating users to answer all questions. Instead, we want to order questions to get information that reduces prediction uncertainty, while not being too burdensome. We illustrate this framework with an example of providing energy estimates to prospective tenants. We also discuss DQO for national surveys and consider connections between our statistics-based question-ordering approach and cognitive survey methodology.