Pseudo Pixel-level Labeling for Images with Evolving Content

TL;DR

This paper introduces a pseudo-labeling technique leveraging image sequences to improve semantic segmentation in domains with scarce annotations, demonstrated on forensic decomposition images with significant accuracy gains.

Contribution

It proposes a novel pseudo-labeling method that propagates annotations across image sequences using unsupervised and CAM-based techniques, reducing manual effort in annotation.

Findings

Improved mean-IoU by up to 3.36% using the proposed method.

Enhanced frequency-weighted-IoU by up to 12.91%.

Effective in domains with evolving content and limited annotations.

Abstract

Annotating images for semantic segmentation requires intense manual labor and is a time-consuming and expensive task especially for domains with a scarcity of experts, such as Forensic Anthropology. We leverage the evolving nature of images depicting the decay process in human decomposition data to design a simple yet effective pseudo-pixel-level label generation technique to reduce the amount of effort for manual annotation of such images. We first identify sequences of images with a minimum variation that are most suitable to share the same or similar annotation using an unsupervised approach. Given one user-annotated image in each sequence, we propagate the annotation to the remaining images in the sequence by merging it with annotations produced by a state-of-the-art CAM-based pseudo label generation technique. To evaluate the quality of our pseudo-pixel-level labels, we train two semantic segmentation models with VGG and ResNet backbones on images labeled using our pseudo labeling method and those of a state-of-the-art method. The results indicate that using our pseudo-labels instead of those generated using the state-of-the-art method in the training process improves the mean-IoU and the frequency-weighted-IoU of the VGG and ResNet-based semantic segmentation models by 3.36%, 2.58%, 10.39%, and 12.91% respectively.