Skip sequencing: A decision problem in questionnaire design

TL;DR

This paper models questionnaire design as a decision problem, focusing on skip sequencing, and proposes a formal framework to optimize the trade-off between cost and informativeness considering data quality issues.

Contribution

It introduces a formal decision-theoretic approach to questionnaire design, specifically analyzing skip sequencing under cost, informativeness, and data quality trade-offs.

Findings

Skip sequencing reduces respondent burden and costs.

Trade-offs between informativeness and data quality are formalized.

Guidelines for choosing survey options considering nonresponse and errors.

Abstract

This paper studies questionnaire design as a formal decision problem, focusing on one element of the design process: skip sequencing. We propose that a survey planner use an explicit loss function to quantify the trade-off between cost and informativeness of the survey and aim to make a design choice that minimizes loss. We pose a choice between three options: ask all respondents about an item of interest, use skip sequencing, thereby asking the item only of respondents who give a certain answer to an opening question, or do not ask the item at all. The first option is most informative but also most costly. The use of skip sequencing reduces respondent burden and the cost of interviewing, but may spread data quality problems across survey items, thereby reducing informativeness. The last option has no cost but is completely uninformative about the item of interest. We show how the planner may choose among these three options in the presence of two inferential problems, item nonresponse and response error.