Ask Me or Tell Me? Enhancing the Effectiveness of Crowdsourced Design Feedback

TL;DR

This paper explores how combining open-ended questions with declarative feedback in crowdsourced design improves feedback neutrality and results in more effective design revisions, addressing sentiment bias issues.

Contribution

It introduces a novel feedback approach that integrates questions with statements and demonstrates its effectiveness through user studies.

Findings

Question-based feedback is more neutral in sentiment.

Question-plus-statement feedback leads to better design revisions.

Combining questions with statements enhances feedback effectiveness.

Abstract

Crowdsourced design feedback systems are emerging resources for getting large amounts of feedback in a short period of time. Traditionally, the feedback comes in the form of a declarative statement, which often contains positive or negative sentiment. Prior research has shown that overly negative or positive sentiment can strongly influence the perceived usefulness and acceptance of feedback and, subsequently, lead to ineffective design revisions. To enhance the effectiveness of crowdsourced design feedback, we investigate a new approach for mitigating the effects of negative or positive feedback by combining open-ended and thought-provoking questions with declarative feedback statements. We conducted two user studies to assess the effects of question-based feedback on the sentiment and quality of design revisions in the context of graphic design. We found that crowdsourced question-based feedback contains more neutral sentiment than statement-based feedback. Moreover, we provide evidence that presenting feedback as questions followed by statements leads to better design revisions than question- or statement-based feedback alone.