Ordinary Search Engine Users assessing Difficulty, Effort, and Outcome for Simple and Complex Search Tasks

TL;DR

This study investigates how ordinary Web users assess difficulty, effort, and outcomes of simple versus complex search tasks, revealing that users struggle more with complex tasks and their judgments are less reliable.

Contribution

It provides empirical insights into user performance and judgment accuracy across different search task complexities, highlighting challenges faced by typical users.

Findings

Judgments are more reliable for simple tasks.

Users find it difficult to assess complexity and effort for complex tasks.

Task complexity influences judgment accuracy.

Abstract

Search engines are the preferred tools for finding information on the Web. They are advancing to be the common helpers to answer any of our search needs. We use them to carry out simple look-up tasks and also to work on rather time consuming and more complex search tasks. Yet, we do not know very much about the user performance while carrying out those tasks -- especially not for ordinary users. The aim of this study was to get more insight into whether Web users manage to assess difficulty, time effort, query effort, and task outcome of search tasks, and if their judging performance relates to task complexity. Our study was conducted with a systematically selected sample of 56 people with a wide demographic background. They carried out a set of 12 search tasks with commercial Web search engines in a laboratory environment. The results confirm that it is hard for normal Web users to judge the difficulty and effort to carry out complex search tasks. The judgments are more reliable for simple tasks than for complex ones. Task complexity is an indicator for judging performance.