# Octopus: A Framework for Cost-Quality-Time Optimization in Crowdsourcing

## TL;DR

Octopus is an AI framework that optimally balances quality, cost, and time in crowdsourcing by using a hierarchical POMDP approach, outperforming existing methods in real-world experiments.

## Contribution

It introduces a novel hierarchical POMDP-based framework to jointly optimize quality, cost, and time in crowdsourcing, addressing a previously intractable multi-objective problem.

## Key findings

- Octopus outperforms state-of-the-art approaches in experiments.
- It effectively manages crowdsourcing tasks in real-world settings.
- Demonstrates practical deployment on Amazon Mechanical Turk.

## Abstract

We present Octopus, an AI agent to jointly balance three conflicting task objectives on a micro-crowdsourcing marketplace - the quality of work, total cost incurred, and time to completion. Previous control agents have mostly focused on cost-quality, or cost-time tradeoffs, but not on directly controlling all three in concert. A naive formulation of three-objective optimization is intractable; Octopus takes a hierarchical POMDP approach, with three different components responsible for setting the pay per task, selecting the next task, and controlling task-level quality. We demonstrate that Octopus significantly outperforms existing state-of-the-art approaches on real experiments. We also deploy Octopus on Amazon Mechanical Turk, showing its ability to manage tasks in a real-world dynamic setting.