# Predicting How to Distribute Work Between Algorithms and Humans to Segment an Image Batch

## TL;DR

This paper introduces a resource allocation framework that predicts the optimal distribution of human and automated efforts for image segmentation tasks, improving segmentation quality across diverse modalities.

## Contribution

The paper presents a novel prediction module and two systems for effectively balancing human and algorithmic segmentation efforts, enhancing image analysis workflows.

## Key findings

- Mix of human and computer efforts outperforms single-resource approaches
- Framework effective across multiple imaging modalities
- Improves quality of both coarse and fine segmentation tasks

## Abstract

Foreground object segmentation is a critical step for many image analysis tasks. While automated methods can produce high-quality results, their failures disappoint users in need of practical solutions. We propose a resource allocation framework for predicting how best to allocate a fixed budget of human annotation effort in order to collect higher quality segmentations for a given batch of images and automated methods. The framework is based on a prediction module that estimates the quality of given algorithm-drawn segmentations. We demonstrate the value of the framework for two novel tasks related to predicting how to distribute annotation efforts between algorithms and humans. Specifically, we develop two systems that automatically decide, for a batch of images, when to recruit humans versus computers to create 1) coarse segmentations required to initialize segmentation tools and 2) final, fine-grained segmentations. Experiments demonstrate the advantage of relying on a mix of human and computer efforts over relying on either resource alone for segmenting objects in images coming from three diverse modalities (visible, phase contrast microscopy, and fluorescence microscopy).