Fusing Forces: Deep-Human-Guided Refinement of Segmentation Masks

TL;DR

This paper introduces a deep neural network that interactively refines segmentation masks with human guidance, significantly reducing manual effort and improving quality in complex annotation tasks like Etruscan mirror illustrations.

Contribution

A novel human-in-the-loop deep learning method for refining segmentation masks that reduces manual effort by up to 75% and enhances quality by up to 26%.

Findings

Achieves expert-level annotation quality with less manual input.

Reduces manual effort required for complex segmentation tasks.

Improves refinement speed and accuracy over purely manual labeling.

Abstract

Etruscan mirrors constitute a significant category in Etruscan art, characterized by elaborate figurative illustrations featured on their backside. A laborious and costly aspect of their analysis and documentation is the task of manually tracing these illustrations. In previous work, a methodology has been proposed to automate this process, involving photometric-stereo scanning in combination with deep neural networks. While achieving quantitative performance akin to an expert annotator, some results still lack qualitative precision and, thus, require annotators for inspection and potential correction, maintaining resource intensity. In response, we propose a deep neural network trained to interactively refine existing annotations based on human guidance. Our human-in-the-loop approach streamlines annotation, achieving equal quality with up to 75% less manual input required. Moreover, during the refinement process, the relative improvement of our methodology over pure manual labeling reaches peak values of up to 26%, attaining drastically better quality quicker. By being tailored to the complex task of segmenting intricate lines, specifically distinguishing it from previous methods, our approach offers drastic improvements in efficacy, transferable to a broad spectrum of applications beyond Etruscan mirrors.