Human Speed-Accuracy Tradeoffs in Search

TL;DR

This paper investigates how users balance the desire for more accurate information against the time spent collecting it on Yahoo Answers, revealing a threshold-based decision process modeled by an inverse Gaussian distribution.

Contribution

It introduces a model of user behavior in information seeking that explains the number of answers received as an inverse Gaussian distribution, validated with real data.

Findings

Users wait longer for additional answers when few answers are received.

The decision to wait follows a threshold-based process.

Number of answers per question fits an inverse Gaussian distribution.

Abstract

When foraging for information, users face a tradeoff between the accuracy and value of the acquired information and the time spent collecting it, a problem which also surfaces when seeking answers to a question posed to a large community. We empirically study how people behave when facing these conflicting objectives using data from Yahoo Answers, a community driven question-and-answer site. We first study how users behave when trying to maximize the amount of acquired information while minimizing the waiting time. We find that users are willing to wait longer for an additional answer if they have received a small number of answers. We then assume that users make a sequence of decisions, deciding to wait for an additional answer as long as the quality of the current answer exceeds some threshold. The resulting probability distribution for the number of answers that a question gets is an inverse Gaussian, a fact that is validated by our data.