Quality Control in Crowdsourced Object Segmentation

TL;DR

This paper investigates methods to improve crowdsourced object segmentation quality by filtering noisy data, detecting unreliable users, and introducing a superpixel-based algorithm that weights user contributions.

Contribution

It introduces new superpixel-based filtering techniques, user reliability detection criteria, and a novel weighted segmentation algorithm that enhances accuracy without prior filtering.

Findings

Filtering improves segmentation accuracy

Detecting bad users increases performance

Weighted contributions yield better results

Abstract

This paper explores processing techniques to deal with noisy data in crowdsourced object segmentation tasks. We use the data collected with "Click'n'Cut", an online interactive segmentation tool, and we perform several experiments towards improving the segmentation results. First, we introduce different superpixel-based techniques to filter users' traces, and assess their impact on the segmentation result. Second, we present different criteria to detect and discard the traces from potential bad users, resulting in a remarkable increase in performance. Finally, we show a novel superpixel-based segmentation algorithm which does not require any prior filtering and is based on weighting each user's contribution according to his/her level of expertise.