SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels

TL;DR

SILT is a novel framework that improves shadow detection by iteratively refining noisy labels using shadow-aware filtering and self-training, leading to significant performance gains over existing methods.

Contribution

The paper introduces SILT, a shadow-aware iterative label tuning method that explicitly handles noisy labels and enhances deep shadow detection models.

Findings

Outperforms state-of-the-art methods by large margins.

Reduces Balanced Error Rate by up to 36.9%.

Effective on multiple datasets including SBU, UCF, and ISTD.

Abstract

Existing shadow detection datasets often contain missing or mislabeled shadows, which can hinder the performance of deep learning models trained directly on such data. To address this issue, we propose SILT, the Shadow-aware Iterative Label Tuning framework, which explicitly considers noise in shadow labels and trains the deep model in a self-training manner. Specifically, we incorporate strong data augmentations with shadow counterfeiting to help the network better recognize non-shadow regions and alleviate overfitting. We also devise a simple yet effective label tuning strategy with global-local fusion and shadow-aware filtering to encourage the network to make significant refinements on the noisy labels. We evaluate the performance of SILT by relabeling the test set of the SBU dataset and conducting various experiments. Our results show that even a simple U-Net trained with SILT can outperform all state-of-the-art methods by a large margin. When trained on SBU / UCF / ISTD, our network can successfully reduce the Balanced Error Rate by 25.2% / 36.9% / 21.3% over the best state-of-the-art method.