TCuPGAN: A novel framework developed for optimizing human-machine interactions in citizen science

TL;DR

This paper introduces TCuPGAN, a 3D segmentation framework that reduces human effort in citizen science by using adversarial learning and LSTM to identify and correct poorly performing slices, significantly decreasing volunteer workload.

Contribution

The paper presents a novel 3D segmentation model with patch-wise adversarial training and an iterative human-machine optimization framework for citizen science applications.

Findings

Reduced volunteer effort by over 60% in a liver tissue segmentation project.

Effective identification of poorly generalized slices for targeted human correction.

Demonstrated scalability and applicability to large 3D datasets in citizen science.

Abstract

In the era of big data in scientific research, there is a necessity to leverage techniques which reduce human effort in labeling and categorizing large datasets by involving sophisticated machine tools. To combat this problem, we present a novel, general purpose model for 3D segmentation that leverages patch-wise adversariality and Long Short-Term Memory to encode sequential information. Using this model alongside citizen science projects which use 3D datasets (image cubes) on the Zooniverse platforms, we propose an iterative human-machine optimization framework where only a fraction of the 2D slices from these cubes are seen by the volunteers. We leverage the patch-wise discriminator in our model to provide an estimate of which slices within these image cubes have poorly generalized feature representations, and correspondingly poor machine performance. These images with corresponding machine proposals would be presented to volunteers on Zooniverse for correction, leading to a drastic reduction in the volunteer effort on citizen science projects. We trained our model on ~2300 liver tissue 3D electron micrographs. Lipid droplets were segmented within these images through human annotation via the `Etch A Cell - Fat Checker' citizen science project, hosted on the Zooniverse platform. In this work, we demonstrate this framework and the selection methodology which resulted in a measured reduction in volunteer effort by more than 60%. We envision this type of joint human-machine partnership will be of great use on future Zooniverse projects.