Herding the Crowd: Automated Planning for Crowdsourced Planning

TL;DR

This paper explores how automated planning can enhance crowdsourced planning tasks by providing oversight and guidance, addressing challenges in interpreting human inputs and steering plans with incomplete models.

Contribution

It conceptualizes automated oversight as primitive planning and discusses opportunities and challenges for integrating sophisticated automated planning into crowdsourcing systems.

Findings

Automated oversight improves human planner effectiveness.

Challenges include interpreting human inputs and incomplete models.

Existing systems vary in handling these challenges.

Abstract

There has been significant interest in crowdsourcing and human computation. One subclass of human computation applications are those directed at tasks that involve planning (e.g. travel planning) and scheduling (e.g. conference scheduling). Much of this work appears outside the traditional automated planning forums, and at the outset it is not clear whether automated planning has much of a role to play in these human computation systems. Interestingly however, work on these systems shows that even primitive forms of automated oversight of the human planner does help in significantly improving the effectiveness of the humans/crowd. In this paper, we will argue that the automated oversight used in these systems can be viewed as a primitive automated planner, and that there are several opportunities for more sophisticated automated planning in effectively steering crowdsourced planning. Straightforward adaptation of current planning technology is however hampered by the mismatch between the capabilities of human workers and automated planners. We identify two important challenges that need to be overcome before such adaptation of planning technology can occur: (i) interpreting the inputs of the human workers (and the requester) and (ii) steering or critiquing the plans being produced by the human workers armed only with incomplete domain and preference models. In this paper, we discuss approaches for handling these challenges, and characterize existing human computation systems in terms of the specific choices they make in handling these challenges.