Adaptive Early-Learning Correction for Segmentation from Noisy Annotations

TL;DR

This paper investigates the learning dynamics of deep segmentation networks trained on noisy data, revealing category-specific early-learning phases, and proposes an adaptive correction method with scale consistency regularization that improves robustness and accuracy.

Contribution

It introduces a novel adaptive early-learning correction approach for segmentation with noisy annotations, leveraging category-specific memorization detection and scale consistency regularization.

Findings

Outperforms standard methods on medical imaging segmentation with synthetic noise.

Achieves state-of-the-art results on PASCAL VOC 2012 with realistic noisy annotations.

Demonstrates category-wise early memorization in segmentation networks.

Abstract

Deep learning in the presence of noisy annotations has been studied extensively in classification, but much less in segmentation tasks. In this work, we study the learning dynamics of deep segmentation networks trained on inaccurately-annotated data. We discover a phenomenon that has been previously reported in the context of classification: the networks tend to first fit the clean pixel-level labels during an "early-learning" phase, before eventually memorizing the false annotations. However, in contrast to classification, memorization in segmentation does not arise simultaneously for all semantic categories. Inspired by these findings, we propose a new method for segmentation from noisy annotations with two key elements. First, we detect the beginning of the memorization phase separately for each category during training. This allows us to adaptively correct the noisy annotations in order to exploit early learning. Second, we incorporate a regularization term that enforces consistency across scales to boost robustness against annotation noise. Our method outperforms standard approaches on a medical-imaging segmentation task where noises are synthesized to mimic human annotation errors. It also provides robustness to realistic noisy annotations present in weakly-supervised semantic segmentation, achieving state-of-the-art results on PASCAL VOC 2012. Code is available at https://github.com/Kangningthu/ADELE